GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications
Abstract
Freehand gesture is an essential input modality for modern Augmented Reality (AR) user experiences. However, developing AR applications with customized hand interactions remains a challenge for end-users. Therefore, we propose GesturAR, an end-to-end authoring tool that supports users to create in-situ freehand AR applications through embodied demonstration and visual programming. During authoring, users can intuitively demonstrate the customized gesture inputs while referring to the spatial and temporal context. Based on the taxonomy of gestures in AR, we proposed a hand interaction model which maps the gesture inputs to the reactions of the AR contents. Thus, users can author comprehensive freehand applications using trigger-action visual programming and instantly experience the results in AR. Further, we demonstrate multiple application scenarios enabled by GesturAR, such as interactive virtual objects, robots, and avatars, room-level interactive AR spaces, embodied AR presentations, etc. Finally, we evaluate the performance and usability of GesturAR through a user study.
BibTeX
@inproceedings{10.1145/3472749.3474769,
author = {Wang, Tianyi and Qian, Xun and He, Fengming and Hu, Xiyun and Cao, Yuanzhi and Ramani, Karthik},
title = {GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications},
year = {2021},
isbn = {9781450386357},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3472749.3474769},
doi = {10.1145/3472749.3474769},
abstract = {Freehand gesture is an essential input modality for modern Augmented Reality (AR) user experiences. However, developing AR applications with customized hand interactions remains a challenge for end-users. Therefore, we propose GesturAR, an end-to-end authoring tool that supports users to create in-situ freehand AR applications through embodied demonstration and visual programming. During authoring, users can intuitively demonstrate the customized gesture inputs while referring to the spatial and temporal context. Based on the taxonomy of gestures in AR, we proposed a hand interaction model which maps the gesture inputs to the reactions of the AR contents. Thus, users can author comprehensive freehand applications using trigger-action visual programming and instantly experience the results in AR. Further, we demonstrate multiple application scenarios enabled by GesturAR, such as interactive virtual objects, robots, and avatars, room-level interactive AR spaces, embodied AR presentations, etc. Finally, we evaluate the performance and usability of GesturAR through a user study.},
booktitle = {The 34th Annual ACM Symposium on User Interface Software and Technology},
pages = {552–567},
numpages = {16},
keywords = {immersive authoring, embodied demonstration, Freehand interactions, Augmented Reality},
location = {Virtual Event, USA},
series = {UIST '21}
}