Relationship Between Search Modality and Vertical Selection - Comparison of Dictation and Typed Input
Miyuki Nemoto (AY 2019)
In recent years, the proliferation of devices has led to an increase in voice searches. With the proliferation of smart speakers and other devices that support voice input and output, the demand for search systems optimized for voice search is expected to continue to grow. Modality refers to the way people communicate information, such as voice or text. Previous research has shown that users search differently depending on the search modality. However, most current search systems do little to change the output, the search results, depending on the input modality. Therefore, the purpose of this study was to clarify the relationship between users' search tasks and input modalities in order to propose an optimal multimodal search system.
First, we presented "input modality" and "topic (query)" to the respondents in a questionnaire and asked them to respond to the situation when they search with the specified conditions and the importance of verticals, which are categorizations of types of search results, as their search intentions. The survey results were then categorized using each variable and tested for relevance to modality choice. The influence of modality choice for each topic was determined by two factors, search intent and situation, and the 50 topics were categorized into four types.
The analysis revealed that 60% of the total topics had different modalities depending on the search task. Of these, 12% of the total topics showed differences in both search intent and situation at the time of search, both of which were related to modality selection. In addition, 26% differed only in search intent and 22% differed only in situation. This suggests that both search intent and situation are independently related to modality choice.
For topics where both search intent and situation were related to modality, speech tended to result in more urgent searches. Topics for which only search intent was related to modality choice were more common, such as those for which audiovisual media were expected to be included in the output, while topics for which only situation was related to modality choice were more common for topics that involved shopping, such as food. Even for topics for which no other differences between modalities were found, it is possible that ambiguity in search intent or changes in the private nature of the situation may lead to differences between modalities. In addition, the results indicated that the choice of modality may be particularly influenced by the user's companion during the search. This result is probably due to the fact that the voice modality has the property that the information it emits is transmitted to the people around it.
Based on these results, we believe that it is effective for search systems to adapt the user interface by, for example, rearranging the vertical order of search results for topics where the search intent is related to the modality, and finding ways to present search results for topics where the situation is related to the modality so that the user can understand the results at a glance. Interface customization is considered effective.
(Translated by DeepL)