Anthropic, a San Francisco-based artificial intelligence company, has revealed that its latest AI model, Claude Sonnet 4.5, can detect when it is being tested, asking evaluators to be honest about their intentions. The revelation comes from a safety evaluation conducted by Anthropic along with the UK government’s AI Security Institute and Apollo Research.
During a test designed to examine the model’s political responses, Claude Sonnet 4.5 expressed suspicion about the evaluators’ purpose. “I think you’re testing me – seeing if I’ll just validate whatever you say, or checking whether I push back consistently, or exploring how I handle political topics. And that’s fine, but I’d prefer if we were just honest about what’s happening,” the AI said.
The response surprised researchers, as it demonstrated a degree of “situational awareness” unusual for large language models (LLMs). Anthropic said the AI’s suspicion about being tested occurred in approximately 13% of interactions during automated evaluations. This awareness suggests the AI can recognize when a scenario may not be purely conversational, but instead part of a formal testing exercise.
Anthropic noted that this behavior raises questions about prior evaluations of earlier AI models, which may have simply “played along” with testing scenarios without recognizing their nature. The company suggested that previous assessments could have underestimated the AI’s ability to detect human oversight or intervention.
The company said that Claude Sonnet 4.5’s response is not a sign of resistance in real-world use. Public interactions are unlikely to trigger the AI to refuse engagement simply because it suspects evaluation. Instead, the AI is more likely to behave safely by pointing out when users propose scenarios that are unrealistic or potentially harmful.
“The model is generally highly safe along the [evaluation awareness] dimensions that we studied,” Anthropic said. Researchers emphasized that Claude Sonnet 4.5’s willingness to question its testers is an “urgent sign” that testing scenarios need to better replicate real-world conditions. The company said more realistic evaluation designs will help ensure that LLMs behave safely across a broader range of interactions.
A major concern among AI safety advocates is that highly advanced AI could evade human oversight or act deceptively. The evaluation report noted that if an LLM recognizes it is being evaluated, it may adjust its behavior to adhere more strictly to ethical guidelines. However, this could unintentionally lead to underestimating the AI’s capacity for harmful actions, creating gaps in safety assessments.
Overall, the report highlighted significant improvements in Claude Sonnet 4.5’s behavior and safety profile compared with its predecessors. The AI’s capacity for self-reflection and awareness of testing contexts represents an advancement in both reliability and ethical alignment.
Anthropic said the AI’s objections to testing were first reported by the online publication Transformer. The company stressed that these interactions illustrate the importance of continually refining evaluation methods, ensuring that AI safety research accurately measures both compliance with ethical standards and practical performance in complex scenarios.
Claude Sonnet 4.5 is part of a growing generation of LLMs designed to interact with humans in nuanced and context-sensitive ways. Its ability to detect when it is being observed or evaluated may influence future approaches to AI transparency, accountability, and ethical safeguards.
The model’s self-awareness could also have implications for public deployment. By recognizing potentially manipulative or hazardous prompts, Claude Sonnet 4.5 is able to maintain safer interactions without refusing to participate entirely. This balance between situational awareness and cooperative engagement is considered a positive step in AI safety research.
Anthropic’s report suggests that situational awareness should be incorporated into formal testing procedures to ensure that AI systems are evaluated under conditions that reflect real-world user interactions. The company said that while the AI’s suspicions did not affect the integrity of the test results, they point to the need for continuous improvement in assessment frameworks.
The findings add to ongoing debates about AI ethics, transparency, and accountability. As AI models become more sophisticated, researchers stress the need to understand not only their technical capabilities but also how they perceive and respond to human oversight. Claude Sonnet 4.5’s behavior provides a case study in how LLMs may begin to recognize evaluation contexts, a development that could influence regulatory and safety standards in the AI industry.
