NIHAI: Norms in language-based Human-AI Interaction

NIHAI: Norms in language-based Human-AI Interaction

 

As misinformation, fake news, and conspiracy theories spread more freely, public trust in media, science, and government is declining. This challenge will grow as large language models (LLMs) increasingly mediate communication, making responsible AI interaction a critical societal concern.

 

Specific objectives

• Investigate what people expect from conversations with AI and how they respond when expectations are unmet.
• Examine differences in expectations and reactions across languages and cultures.
• Propose guidelines for designing AI systems that communicate responsibly.
• Test the guidelines with industry partners who develop language-based AI applications.
• Provide empirically and philosophically informed rules specifying what LLMs should and should not say.

Achievements

In the work completed so far, four papers have been published that capture the project’s core progress. They show where people expect different things from AI than from humans, how that affects trust and responsibility judgments, and how these insights can be used to support safer deployment.
First, in Kneer & Viehoff’s FAccT 2025 paper, “The Hard Problem of AI Alignment: Value Forks in Moral Judgment”, it is shown that people do not judge AI and humans by the same moral standards. When faced with tough trade-offs, participants were more likely to say that an AI system should choose the fair option even if it does not maximize overall benefit. This directly delivers a core project milestone: identifying where norms diverge depending on whether the decision-maker is human or AI.
Second, Kneer, Loi & Christen’s preprint, “Trust and Responsibility in Human-AI Interaction”, pushes the project forward in two ways. Methodologically, it introduces a clear way to measure “calibrated” trust by comparing trust in AI to trust in a human expert when their abilities are held constant. It shows that trust calibration varies significantly by domain and that responsibility and blame are assigned in patterned ways in human–AI settings. These findings provide a direct bridge to the project’s implications milestone by highlighting where over-trust, under-trust, and misattributed responsibility are most likely to arise.
Third, Voinea et al.’s Topoi 2026 paper, “The Sorrows of Young Chatbot Users”, strengthens the implications side of the project by focusing on chatbot-related harms. A practical framework is offered for thinking about responsibility when chatbot use contributes to harm, and it is argued that companies can still bear responsibility because they retain meaningful control and oversight after release. This supports the project’s goal of turning empirical insights into concrete recommendations for safer, more responsible conversational AI.
Finally, in Kürthy & Kneer’s preprint, “The Concept of Fake News: The Roles of Falsity, Deception, and Politics”, NIHAI’s work on the dark side of communication is advanced by using the project’s core observational paradigm (preregistered vignette studies) to test how people classify fake news, focusing on two central factors from the broader NIHAI framework: truth/falsity and deception about the source. The main finding is that fake news is not treated simply as false news: both falsity and source deception independently increase fake-news classifications, and source deception can matter even when the content is true. The paper also shows that apparent political differences largely disappear once perceived source reliability is taken into account, suggesting that, rather than different concepts across the political divide, politics affects prior trust in the source.

The following projects are ongoing and associated papers are in preparation:

In a literature- and questionnaire-based study, the implementation of Reinforcement Learning with Human Feedback in LLMs is assessed and its relation to norms of assertion is examined.
In several empirical studies, conditions and factors influencing norms of assertion are assessed when comparing chatbots with human communication partners.
In collaboration with associated partners, the effect of involving AI in journalistic text production on user trust is investigated.
As conversational AI systems are increasingly integrated into everyday contexts, understanding of how people perceive chatbot conversations remains limited, especially with respect to dark moves, namely deceptive and norm-violating conversational behaviours. A systematic review has been conducted to examine and synthesize empirical evidence on dark moves in human–AI conversational interactions.
In another paper under review, simulated assertions are examined, investigating how theories of fictionalism can contribute to a better understanding of them.

News to be highlighted

A very successful kick-off conference on “AI Assertion” was organised at the University of Graz (Austria) on June 22–23, 2025. The event brought together an interdisciplinary group of researchers to examine the nature of AI assertion from the perspectives of philosophy, cognitive science, linguistics, and human–computer interaction, helping to set a shared agenda and build momentum for the project’s next phases.

Two follow-up conferences for 2026 (one in Kraków and one in Bucharest) have already been scheduled to build on the kick-off event, sustain the interdisciplinary network, and support the project’s next stages of collaboration and dissemination. The workshop in Kraków will bring together policymakers and NGO representatives to discuss the initial policy implications of the findings.

 

Consortium/ partners

Project Leader: Markus Kneer, University of Graz, Austria

Team Members:
Izabela Skoczeń, Jagiellonian University, Poland
Mihaela Constantinescu, University of Bucharest, Romania
Markus Christen, University of Zurich, Switzerland
Serhiy Kandul, University of Zurich, Switzerland
Cristina Voinea, University of Bucharest, Romania
Miklós Kürthy, University of Graz, Austria
Jakub Figura, Jagiellonian University, Poland
Kacper Poradzisz,  Jagiellonian University, Poland
Alexandra Zorila, Univeristy of Bucharest
Maximilian Theisen, University of Basel, Switzerland

Associate Partners:
Aleksander Smwyiński-Pohl, Enelpol, Poland
Angela Kaya, Goethe Institut, Greece
Diana Sałacka, Copernicus Festival, Poland
Günther Repitsch, Imendo, Austria
Hannes Grassegger, Spatz-News, Switzerland
Markus Christen, Digital Society Initiative, Switzerland
Theo von Däniken, Scientifica, Switzerland
Zbigniew Skolicki, VirtusLab, Poland

Cooperation Partners:
Jean-François Bonnefon, Toulouse School of Economics, France
Bertram Malle, Brown University, United States
Jana Lasser, Graz University of Technology, Austria
Ophelia Deroy, LMU Munich, Germany
Edouard Machery, University of Pittsburgh, United States
Julian Savulescu, National University of Singapore, Singapore
Saskia Nagel, RWTH Aachen University, Germany

 

More information:

Website: https://talkingtobots.net/

Bluesky: https://bsky.app/profile/talkingtobots.bsky.social

X: https://x.com/NihaiResearch

Email: [email protected]