Anthropic announces an evaluation conducted from the inside with Faculty, a specialised subsidiary of Accenture. The access promised to the work raises a decisive question: how can oversight be made independent of its funding?
Evaluating a model once it has been delivered leaves out of scope some of the choices that shaped it. Anthropic and Accenture announce an arrangement in which evaluators could follow the work from inside the company. Faculty, Accenture’s AI-specialised business, is to carry out the mission.
Access changes the scope of oversight
The 18 September announcement provides for model evaluations, adversarial testing and a review of safeguards. Anthropic describes access comparable to that of an employee. The two companies each envisage investing at least one billion dollars over five years to develop this capability.
The company specifies, however, that the terms remain to be defined and that it will directly fund Accenture’s work. It also indicates that the partnership is not exclusive. These caveats are essential: an announced arrangement is not yet an oversight system whose effectiveness has been demonstrated.
For a house, asking for usable proof
A management that entrusts archives, processes or decisions to AI can draw a practical question from this development: what information is a third party actually able to verify? An evaluator’s name is not enough. Its access, the situations it can examine and its freedom to report determine the reach of the oversight.
The same requirement applies at the level of an internal project. A committee must be able to distinguish the tests carried out by the vendor, those conducted by the user team and independent verifications. Each answers a different need.
Embedded evaluation could reduce certain blind spots tied to outside observation. Its credibility will also depend on the safeguards that protect the evaluator’s judgment. Access to evidence and the ability to make it public are two distinct conditions; the development of the partnership will show how they fit together.
Cette publication est également disponible en :
