AI Safety, Alignment & Governance

Overview

As frontier AI models and agents have grown more capable, a distinct ecosystem of technical safety researchers and government evaluators has emerged to test them before and after deployment. METR conducted a pilot rogue-deployment risk assessment across Anthropic, Google, Meta and OpenAI in 2026, finding internal AI agents plausibly had the means, motive and opportunity for small-scale rogue actions though not larger ones; Redwood Research partnered with the UK AI Security Institute on 'AI control' safety cases and advises Google DeepMind and Anthropic on misalignment mitigation; the US Center for AI Standards and Innovation (CAISI, formerly the US AI Safety Institute) expanded pre-deployment testing agreements to five frontier labs and completed over 40 model evaluations by mid-2026; the UK AI Security Institute evaluated Anthropic's Claude Mythos as too dangerous to release in its tested form; and the EU AI Act's chatbot-transparency and high-risk-system rules became enforceable on August 2, 2026 with penalties up to 7% of global revenue.

Strategic Connections

Strategic Risks

• Fragmented, jurisdiction-specific evaluation regimes (US CAISI, UK AISI, EU AI Act) create compliance complexity and potential regulatory arbitrage for frontier labs
• Political turnover at safety institutes (CAISI's director resigning three months into the role) creates continuity and independence risk for evaluation programs

Future Outlook

• Continued expansion of pre-deployment testing agreements as more jurisdictions stand up frontier-model evaluation capacity
• EU AI Act's August 2026 enforcement wave likely to become a de facto global compliance baseline given the size of the EU market

Sources

2 sources, 50% average source confidence — full citations in the Full Analyst Report.