Effect of Pesonality Trais of Large Language Models on Information Retrieval Behavior
Yuta Imasaka (AY 2024)
LLMs have been leveraged across a wide range of natural language processing tasks. In the field of IR, LLMs have begun to substitute for human efforts in various activities, such as query formulation and relevance judgment. However, while prior studies indicate that individual traits influence human information-seeking behaviors, it remains insufficiently understood how LLMs assigned with personality traits behave in IR scenarios and how their performance compares. In this work, we propose a method for assigning LLM agents with five personality dimensions based on the Big Five model, evaluate the effectiveness of this approach using the IPIP-NEO-120, and then conduct two experiments - query generation, and relevance judgment - to assess both their behavioral characteristics and performance. The results suggest that personality traits partially affect query length, lexical choices, and relevance judgment behavior, although no pronounced differences in retrieval effectiveness or relevance judgment performance were observed. These findings imply the possibility of incorporating personality traits into LLM applications without substantially degrading IR performance. Future research focusing on more advanced IR tasks, such as query reformulation and reranking, is expected to enhance our comprehensive understanding of how personality traits influence LLM-driven information-seeking behavior.