GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications (UIST2021)
[Honorable Mention]

Tianyi Wang*, Xun Qian*, Fengming He, Xiyun Hu, Yuanzhi Cao, and Karthik Ramani. "GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications." In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. (UIST 2021) DOI: https://doi.org/10.1145/3472749.3474769

Abstract: Freehand gesture is an essential input modality for modern Augmented Reality (AR) user experiences. However, developing AR applications with customized hand interactions remains a challenge for end-users. Therefore, we propose GesturAR, an end-to-end authoring tool that supports users to create in-situ freehand AR applications through embodied demonstration and visual programming. During authoring, users can intuitively demonstrate the customized gesture inputs while referring to the spatial and temporal context. Based on the taxonomy of gestures in AR, we proposed a hand interaction model which maps the gesture inputs to the reactions of the AR contents. Thus, users can author comprehensive freehand applications using trigger-action visual programming and instantly experience the results in AR. Further, we demonstrate multiple application scenarios enabled by GesturAR, such as interactive virtual objects, robots, and avatars, room-level interactive AR spaces, embodied AR presentations, etc. Finally, we evaluate the performance and usability of GesturAR through a user study.