Anthropic is letting METR researchers look under the hood with employee-level access
By Pradhyuman,
Anthropic is giving employee-level access to six researchers from nonprofit organization METR. The researchers receive company laptops, office space, and internal tools to independently assess Anthropic models and safety practices.
Anthropic stated that it wants public accountability and aims to close the gap between external evaluators and internal teams. The company compared the arrangement to bank regulators placing resident examiners inside major financial institutions.
METR publishes its own assessments regarding risks and incidents. Anthropic relinquished the right to veto publication, retaining only the ability to require redactions for customer data, attorney-client privilege, and model architectures. METR is permitted to publicly evaluate whether those redactions are justified.
The arrangement is part of a three-tier evaluation system. METR occupies the top tier, while bottom-tier evaluators require Anthropic approval to publish findings. Anthropic CEO Dario Amodei proposed embedding outside evaluators in a September essay.
Maintained by Pradhyuman.
Filed under: Anthropic, Models & Research, Policy & Safety