UbiComp '19: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

ScratchThat: Supporting Command-Agnostic Speech Repair in Voice-Driven Assistants

Jason Wu, Karan Ahuja, Richard Li, Victor Chen, Jeffrey Bigham

PDFpaperLinkpublication

Abstract

Speech interfaces have become an increasingly popular input method for smartphone-based virtual assistants, smart speakers, and Internet of Things (IoT) devices. While they facilitate rapid and natural interaction in the form of voice commands, current speech interfaces lack natural methods for command correction. We present ScratchThat, a method for supporting commandagnostic speech repair in voice-driven assistants, suitable for enabling corrective functionality within third-party commands. Unlike existing speech repair methods, ScratchThat is able to automatically infer query parameters and intelligently select entities in a correction clause for editing. We conducted three evaluations to (1) elicit natural forms of speech repair in voice commands, (2) compare the interaction speed and NASA TLX score of the system to existing voice-based correction methods, and (3) assess the accuracy of the ScratchThat algorithm. Our results show that (1) speech repair for voice commands differ from previous models for conversational speech repair, (2) methods for command correction based on speech repair are significantly faster than other voice-based methods, and (3) the ScratchThat algorithm facilitates accurate command repair as rated by humans (77% accuracy) and machines (0.94 BLEU score). Finally, we present several ScratchThat use cases, which collectively demonstrate its utility across many applications.

Citation

Wu, J., Ahuja, K., Li, R., Chen, V., & Bigham, J. (2019). ScratchThat: supporting command-agnostic speech repair in voice-driven assistants. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(2), 1-17.

BibTeX

@article{wu2019scratchthat,
title={ScratchThat: supporting command-agnostic speech repair in voice-driven assistants},
author={Wu, Jason and Ahuja, Karan and Li, Richard and Chen, Victor and Bigham, Jeffrey},
journal={Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
volume={3},
number={2},
pages={1--17},
year={2019},
publisher={ACM New York, NY, USA}
}