Losing its Touch: Understanding User Perception of Multimodal Interaction and Smart Assistance
Margarita Esau, Veronika Krauß, Dennis Lawo, Gunnar Stevens
DIS'22: ACM SIGCHI Conference on Designing Interactive Systems (DIS)
Session: Multisensory design
Abstract
Intelligent Personal Assistants (IPA) are advertised as reliable companions in the everyday life to simplify household tasks. Due to speech-based usability issues, users struggle to deeply engage with current systems. The capabilities of newer generations of standalone devices are even extended by a display, also to address some of the weaknesses like memorizing auditive information. So far, it is unclear how the potential of a multimodal experience is realized by designers and appropriated by users. Therefore, we observed 20 participants in a controlled setting, planning a dinner with the help of an audio-visual-based IPA, namely Alexa Echo Show. Our study reveals ambiguous mental models of perceived and experienced device capabilities leading to confusion. Meanwhile, the additional visual output channel could not counterbalance the weaknesses of voice interaction. Finally, we aim to illustrate users' conceptual understandings of IPAs and provide implications to rethink audio-visual output for voice-first standalone devices.
DOI:: https://doi.org/10.1145/3532106.3533451
WEB:: https://dis.acm.org/2022/
Video Presentations of DIS 2022