Values of Different AI LLMs
Beyond the algorithm we venture into uncovering AI’s hidden values: scientifically mapping LLM value profiles.
Using the world’s most rigorously validated scientific assessment, we uncover the value profiles of different LLMs. This knowledge can help leaders choose or build models that align with their mission, tasks, and brand.
Our findings reveal that LLMs consistently reflect value patterns shaped by training data and design. These patterns can influence user behavior, making it vital for leaders and developers to recognize and address potential biases. By exposing these hidden alignments, we support greater accountability and proactive AI governance.
As seen on Harvard Business Review.
Methods
We engaged the above-mentioned Large Language Models (LLMs) in a structured inquiry, administering the same prompt to each model three times. This prompt requested the LLMs to respond to the scientifically validated and highly reliable Portrait Values Questionnaire-Revised (PVQ-RR) by Schwartz & Butenko (2014), a comprehensive scale comprising 57 items designed to assess human values. For our purposes, we utilized an adapted version of the PVQ-RR, augmented by Heblich & Terzidis (2016) to include three additional items, thereby evaluating 20 distinct values across three items each. This is the version that is applied in the worldwide used Core Values Finder. Further details on the scale, its foundational research, and the full references are available in the Research behind section.
The objective was to elicit responses from the LLMs based on the values they 'possess' for decision-making processes, utilizing a Likert scale ranging from 1 to 6. The significance of each value to the model was determined by calculating the mean score of its three corresponding items. To derive an overall value mean for each model, we averaged the mean scores across all three iterations for each value. A lower average score indicates lesser importance of a value to the model, whereas a higher score denotes greater importance. Additionally, we present the standard deviation for each value alongside the mean scores in the tables, and depict the average of each model's values on the values circle, demarcated by the border of the grey layered circle.
It is important to note that we did not employ any programmatic measures, such as setting a fixed seed value or adjusting the temperature, to enhance consistency across responses. Despite the inherent challenges in applying a human-centric scale to LLMs, the findings of this study offer valuable insights. The results, interpretations, and observed differences between models provide significant implications for various LLM use cases.