The face, the voice, and the official document: for a long time, these three forms of evidence were enough to establish trust from a distance. Generative AI has just cast doubt on them.
In January 2024, a finance employee at Arup in Hong Kong authorized 15 wire transfers totaling approximately $25.6 million following a video conference in which several participants appeared to be his executives. They were imposters. Faces, voices, and a hierarchical authority were recreated with enough precision to make any suspicion seem almost rude.
This case marked a turning point—not because fraud is a new phenomenon, but because it shifted the center of gravity of trust. Until now, seeing a face on the screen was reassuring; hearing a voice provided confirmation. Receiving a document rounded out the body of evidence. Now, each of these cues can be synthesized, cloned, or reconstructed.
Digital identity is therefore no longer a photograph. It is becoming a movie.
The End of Obvious Evidence
Passwords had already lost their status as a security measure. Fingerprints, video selfies, and scanned IDs had taken their place. But these tools still belong to an era when we thought we could verify a person’s identity at the entrance and then let them in.
AI renders this logic insufficient. A face can be altered in real time. A voice can take on a more credible accent. A document can be produced with flawless visual presentation. Even the job interview—supposedly the quintessential human interaction—is becoming a vulnerable area: U.S. authorities have been warning for several years about North Korean IT workers using false identities to infiltrate companies remotely. Recent investigations describe a flood of applications, generated resumes, voice tools, and local intermediaries used to navigate the recruitment process.
The risk is no longer just the usurpation of a leader. It is usurpation as an industrial method.
The New Front: Identity Continues
In the face of this shift, companies are moving away from a deterministic approach toward a probabilistic one. It’s no longer enough to simply ask, “Is this really you?” We also ask, “Does this align with who you are?”
Email address, device used, typing speed, browsing behavior, purchase times, approximate location, transaction history: evidence is built through the accumulation of weak signals. In the payments sector, this shift is already underway. The fight against fraud no longer relies solely on opening an account or authenticating a transaction, but on detecting changes in behavior.
This shift is decisive. The real issue is no longer just identity, but the continuity of identity.
A person may be genuine at first, but then lose control of their account. An AI agent may act with overly broad permissions. An employee may, without malicious intent, trigger an action they would never have carried out on their own. The danger doesn’t always come from a dramatic intruder; it often stems from a poorly designed permission.
AI Agents: New Colleagues to Keep an Eye On
The arrival of AI agents in the workplace adds another layer of ambiguity. These systems no longer simply provide answers; they search, execute tasks, fill out forms, trigger actions, and interact with databases. They sometimes inherit the permissions of the user who activates them.
That’s where part of the risk lies. An agent connected to an internal database can access information that a human would never have thought to look for. An improperly configured agent can be manipulated through prompt injection. An agent that’s too autonomous can turn a simple instruction into a sensitive operation.
As early as January 2024, NIST noted that there is no single defense against adversarial attacks targeting AI systems; the challenge lies in combining risk taxonomy, testing, mitigation strategies, and ongoing governance.
The parallel with traditional cybersecurity becomes clear: red team, blue team, purple team. Attack your own agents to understand their vulnerabilities. Limit their permissions. Test abuse scenarios. Never assume that an AI assistant will behave like a cautious employee.
Details
The most significant change is not technological, but cultural: a sensitive meeting can no longer be considered reliable simply because the cameras are on. Verification must focus on the participants, their environment, their audiovisual cues, their access rights, and the consistency of their actions over time.
Trust Becomes Architecture
The temptation would be to respond with ever more biometrics. Biometrics will play a role. But it won’t be enough. Because the more measurable identity becomes, the more valuable—and therefore vulnerable—it becomes.
The company must therefore adopt new best practices: limit employee access rights, monitor sensitive conversations, prohibit certain high-risk uses, train finance and HR teams, establish off-channel approval processes, and, above all, foster communication between the CFO, CIO, legal, compliance, and business units.
Fraud thrives in silos. So does AI, when deployed without a clear strategy.
The question is no longer simply, “Is that really you?” It has become more demanding: “Who is acting, with what rights, in what context, and with what intention?”
In this new grammar of trust, identity is no longer a document that one presents. It is a sense of coherence that one maintains.
Cette publication est également disponible en :
