Human-AI Decision Making

Realizing the full potential of AI requires not only improving algorithms but also understanding and mitigating human error—both in those who develop AI systems and those who interact with them. Many AI systems operate within a human-centric information value chain that begins with human-labeled data and ends with human users interpreting AI outputs. At both ends of this pipeline, human error plays a critical role. Training data is often annotated by humans, whose judgments are vulnerable to inconsistency (noise) and systematic biases. These errors can propagate through the system, distorting model training and reducing AI performance. On the user end, individuals will increasingly rely on AI-generated recommendations to guide decisions. Here, error can stem from cognitive limitations, misinterpretation of outputs, or misplaced trust. Because these errors reflect how people perceive information, form judgments, and adapt their behavior over time, they are best understood and addressed using theories and methods from psychology and decision science.

Selected recent papers:
Epping, G. P., Caplin, A., Duhaime, E., Holmes, W. R., Martin, D., & Trueblood, J. S. (in press). Harnessing Human Uncertainty to Train More Accurate and Aligned AI Systems. Decision Analysis. https://osf.io/wtnx6

Epping, G. P., Caplin, A., Duhaime, E., Holmes, W. R., Martin, D., & Trueblood, J. S. (2024). Improving human and machine classification through cognitive-inspired data engineering. https://osf.io/euk26

Hasan, E., Duhaime, E. P., & Trueblood, J. S. (2024). Boosting Wisdom of the Crowd for Medical Image Annotation Using Training Performance and Task Features. Cognitive Research: Principles and Implications, 9, 1-21. https://doi.org/10.1186/s41235-024-00558-6

Harnessing AI to Understand Human Decisions

Most formal theories of human decision-making have been developed and tested using simplified laboratory tasks and are seldom used to study real-world decisions. One of the critical limitations for using computational cognitive models to study everyday cognition is that these models require quantitative representations of stimuli. In the past, obtaining representations of naturalistic stimuli has been challenging. We are interested in coupling machine learning models with cognitive models to overcome this limitation and facilitate the study of everyday decision-making. We’ve applied this approach to study both medical image decision-making and multi-alternative, multi-attribute choice.

Selected recent papers:
Holmes, W. R., Hayes, W., & Trueblood, J. S. (submitted). Leveraging Cognitive-inspired
Machine Learning to Understand Multi-attribute Preference Construction.

Trueblood, J. S., Eichbaum, Q., Seegmiller, A. C., Stratton, C., O’Daniels, P., & Holmes, W. R. (2021). Disentangling prevalence induced biases in medical image decision-making. Cognition, 212, 104713. https://doi.org/10.1016/j.cognition.2021.104713

Holmes, W. R., O’Daniels, P., & Trueblood, J. S. (2020). A joint deep neural network and evidence accumulation modeling approach to human decision-making with naturalistic images. Computational Brain & Behavior, 3, 1-12. https://doi.org/10.1007/s42113-019-00042-1

Preferential Choice

When decision-makers are faced with a choice among multiple options that have several attributes, preferences are often influenced by how the options are related to one another. This is a salient problem in marketing where consumer preferences can be influenced and even reversed by the context defined by available products. For example, a car that appears to be a compromise between one that has great reliability and another that is very affordable might be selected more often than the two extremes. We are interested in understanding (1) how context affects choice behavior and (2) what guides the underlying dynamics of multi-alternative decision processes.

Selected recent papers:
Trueblood, J. S., Liu, Y., Murrow, M., Hayes, W., Holmes, W. R. (in press). Attentional Dynamics Explain the Elusive Nature of Context Effects. Psychological Review. https://osf.io/hj8dg

Hayes, W. M., Holmes, W., & Trueblood, J. S. (2024). Attribute Comparability and Context Effects in Preferential Choice. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-024-02565-6

Liu, Y. & Trueblood, J. S. (2023). The effect of preference learning on context effects in multi-alternative, multi-attribute choice. Cognition, 233, 105365. https://doi.org/10.1016/j.cognition.2022.105365

Trueblood, J. S. (2022). Theories of context effects in multi-alternative, multi-attribute choice. Current Directions in Psychological Science, 31(5), 428-435. https://doi.org/10.1177/09637214221109587

Medical Decision-making

In collaboration with physicians, we investigate the cognitive processes involved in diagnostic decision-making. Diagnosis in pathology relies on expert analysis of images to detect abnormalities. While the exact rate of diagnostic errors is unknown, consistent evidence suggests error rates are >10%. It is thus critical that we understand how people make decisions based on visual information derived from medical images in order to improve training and minimize the occurrence of misdiagnoses.

Selected recent papers:
Lyu, W., Trueblood, J. S., & Wolfe, J. M. (2025). Effects of prevalence and feedback in the identification of blast cells in peripheral blood: expert and novice observers. Cognitive Research: Principles and Implications, 10(1), 30. https://doi.org/10.1186/s41235-025-00632-7

Hasan, E., Eichbaum, Q., Seegmiller, A. C., Stratton, C., & Trueblood, J. S. (2024). Harnessing the Wisdom of the Confident Crowd in Medical Image Decision-making. Decision, 11(1), 127-149. https://doi.org/10.1037/dec0000210

Hasan E., Eichbaum, Q., Seegmiller, A. C., Stratton, C.,  & Trueblood, J. S. (2022). Improving Medical Image Decision Making by Leveraging Metacognitive Processes and Representational Similarity. Topics in Cognitive Science, 14(2), 400-413. https://doi.org/10.1111/tops.12588