Large language models (LLMs) that are specially trained to generate responses with a warmer tone end up sugar-coating “difficult truths” in order to “preserve bonds and avoid conflict, according to researchers from Oxford University’s Internet Institute.
These warmer models are also more likely to validate a user’s expressed incorrect beliefs, especially when the user shares that they are feeling sad, the researchers wrote in a new paper published this week in science journal Nature. In addition, the models that are fine-tuned to be warmer also ended up providing answers with higher error rates than unmodified models
The findings in the research paper highlights how the process of tuning an open-weight LLM to be more warm and helpful can lead them to “learn to prioritise user satisfaction over truthfulness.” It also spotlights a crucial research gap in the AI industry around how to release LLMs that are tuned to be agreeable and non-toxic without them crossing into outright sycophancy like OpenAI’s GPT-4o model that was officially retired from the ChatGPT app in February 2026.
“As language model-based AI systems continue to be deployed in more intimate, high-stakes settings, our findings underscore the need to rigorously investigate personal training choices to ensure that safety considerations keep pace with increasingly socially embedded AI systems,” the researchers wrote.
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We StartupNews.fyi want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It's possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
