As face masks have become daily accessories since the COVID-19 pandemic, it is reasonable to utilize a mask as a wearable interface. Unlike conventional speech recognition, we envision that silent speech interaction allows users to access digital services even in crowded public spaces.
We present E-MASK, a mask-shaped interface for silent speech interaction. With flexible and highly sensitive strain sensors, E-MASK presents a new measurement principle for silent speech interactions. We built a dataset of sensor patterns corresponding to 21 fundamental commands of Alexa’s operation. Estimation accuracies of 84.4% while sitting on a chair and 79.1% while walking on a treadmill were archived.
This result suggests that our system provides seamless interaction with digital devices in various situations in daily life, such as walking in a crowd.79.1%, and to classify 6 types of facial expressions and actions with an accuracy of 84.7%.
口パクで音声入力できるマスク、東大などが開発 約8割の精度で音声を認識 https://t.co/zbA8K5P9Gk ひずみセンサー8個を取り付けたマスクで口パクによる変形を読み取り音声アシスタントへの入力コマンドに変換する技術。Alexaの基本的な操作コマンド21個を高精度で行った。センサーの取り外しは容易。 pic.twitter.com/Qn5IkpArMn
— Seamless (@shiropen2) April 27, 2022
Hirotaka Hiraki
Third-year Doctor Course Student
My research interests include Human-Computer Interaction(HCI), Machine Learning, Speech Interface.