Emojilization: An Automated Method For Speech to Emoji-Labeled Text

Hu, J., et al. (2019). Emojilization: An Automated Method For Speech to Emoji-Labeled Text. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM. https://doi.org/10.1145/3290607.3313071

Abstract

Speech To Text (STT) plays a significant role in Voice User Interface (VUI). While preserving necessary semantic information in converted text, STT generally captures no or limited emotional information. In this paper, we present an emojilization tool to automatically attach related emojis to the STT-generated texts by analyzing both textual and acoustic features in speech signals. For a given voice message, the tool selects the most representative emoji from 64 most commonly used emojis. We conducted a pilot study with 34 participants. In our study, 159 utterances were labeled with emojis by our tool. The emotion restoration effect was evaluated. The results indicate that the proposed tool effectively compensates for emotion loss.