AI, Deepfakes and the New Crisis of Verification
Synthetic voices, images and video can manufacture familiarity at scale. The answer is not universal suspicion, but verification practices that remain dependable when appearances are convincing.
For most of modern life, seeing and hearing have functioned as forms of proof.
If we recognised a person’s voice on the telephone, we assumed it was them. If we saw someone speaking in a video, we treated the footage as evidence. A familiar face, a known accent or an apparently live conversation helped us decide who was real and what could be trusted.
That assumption is becoming less reliable.
Artificial intelligence can now generate convincing messages, reproduce voices, alter images and create videos of people appearing to say things they never said. The technology does not need to produce a flawless imitation. It only needs to be believable for long enough to influence a decision.
This is creating a new crisis of verification. The problem is not simply that fake content exists. Manipulated photographs, fabricated stories and impersonation are not new. What has changed is the speed, scale and accessibility with which convincing material can be produced—and the ease with which it can be combined with personal information already available online.
The result is an environment in which familiarity can be manufactured.
The Scam Is Familiar. The Production System Is New.
It is tempting to describe AI-enabled fraud as an entirely new category of crime. In practice, many of the underlying tactics are well established.
Scammers still impersonate trusted people. They still create urgency, borrow authority, exploit emotion and discourage independent checking. They still want a target to transfer money, reveal information, surrender access or make a decision before doubt has time to develop.
AI does not replace these methods. It strengthens them.
A generic fraudulent message can be rewritten to sound natural, professional or emotionally convincing. Public information can be used to personalise a story. A short audio sample may help create a synthetic voice. A photograph can be animated. A video call can be manipulated. Translation tools can allow the same operation to approach people across languages and regions.
None of this requires the output to withstand prolonged forensic examination. Fraud takes place in moments of pressure. A distressed voice message, an instruction apparently sent by a senior executive or a brief video call may be enough to lower resistance.
The criminal advantage lies in reducing the time and cost required to create credibility.
When Familiarity Can Be Manufactured
Human beings do not verify every interaction from first principles. We use signals.
We notice a voice, face, writing style, job title, telephone number, email address or shared detail. We compare these signals with what we already know and make a rapid judgement. This is usually necessary. Daily life would become impossible if every conversation required a formal identity check.
AI-enabled impersonation exploits this practical reliance on recognition.
Imagine receiving a call from someone who sounds like a relative. They are upset. They say they have lost their telephone, are using another number and need money urgently. The request fits the emotional tone of an emergency. The voice appears to confirm the identity. The unfamiliar number is explained before it can become a warning sign.
Or imagine an employee joining an online meeting in which a person who looks and sounds like a senior colleague requests a confidential document or urgent payment. The employee is not simply responding to an image. They are responding to hierarchy, workplace expectations and the fear of delaying an important instruction.
The deception works because several signals support one another. The face validates the voice. The voice validates the story. The urgency explains why normal procedure should be bypassed.
Once synthetic media enters this sequence, recognition can no longer carry the weight of verification on its own.
A Deepfake Does Not Have to Be Perfect
Public discussion often focuses on whether a deepfake can be detected by looking for unusual blinking, distorted hands, mismatched lighting or unnatural speech. These clues may sometimes be useful, but they are a weak foundation for safety.
The technology changes quickly. Compression, poor connections and low-quality recordings can make authentic media look suspicious, while a convincing fake may contain no obvious defect. A person under pressure may also have neither the time nor the specialist knowledge to examine the material carefully.
More importantly, a scammer controls the conditions of the interaction.
They can keep a call short. They can claim the signal is weak. They can use emotion to interrupt questions. They can combine genuine and synthetic material. They can choose a channel in which visual or audio imperfections seem normal.
The right question is therefore not only, “Does this look real?”
It is, “Can I verify this request through a separate, trusted route?”
That shift matters. It moves the burden away from trying to become an expert in synthetic media and towards using a process that remains useful even as the technology improves.
The Personalisation of Deception
The information required to make a scam persuasive is often already available.
Professional biographies identify employers, responsibilities and senior relationships. Social media reveals family connections, travel, interests, celebrations and periods of vulnerability. Company websites publish names, photographs, events and contact details. Audio and video appear in webinars, interviews, presentations and personal posts.
Individually, these details may appear harmless. Combined, they can help a criminal build a plausible approach.
AI can assist with organising this material and turning it into tailored communication. A message can refer to a real colleague, recent event or shared interest. Its tone can be adapted to a particular profession or relationship. It may contain enough accurate information to make the false part feel credible.
This does not mean people should disappear from public life or treat every online contribution as dangerous. Visibility is essential to professional, creative and civic participation. The responsibility for fraud does not belong to the person whose public voice, image or information is misused.
It does mean that personal knowledge should no longer be mistaken for proof of identity. A caller knowing a family member’s name, a manager’s schedule or a private-sounding detail does not establish that the caller is genuine.
In an age of accessible information, knowledge can be copied. Identity must be verified.
The Emotional Power of a Familiar Voice
Voice cloning deserves particular attention because voices carry more than words. They communicate identity, emotion, age, accent, intimacy and authority.
We may respond to a familiar voice before we have consciously evaluated the request. A parent hearing a frightened child, an employee hearing a forceful executive or a community member hearing a trusted public figure is placed into a relationship, not merely presented with information.
That relationship can trigger care, obedience, fear or loyalty.
Scammers understand that emotion narrows attention. A demand for immediate action can prevent a person from noticing inconsistencies or making a separate call. The target may also worry that verification will appear uncaring, disrespectful or insubordinate.
This is why effective protection must give people social permission to check.
Families can agree in advance that an unexpected financial request will always be verified through another method. Organisations can make clear that no seniority level exempts a request from security procedures. Communities can normalise a pause without framing caution as distrust.
A good verification culture does not weaken relationships. It protects them from imitation.
The Risk of Trusting Nothing
There is another danger in the spread of synthetic media: genuine evidence may be dismissed simply because fabrication is possible.
A real recording can be described as AI-generated. Authentic testimony can be challenged without serious examination. Public confidence can be weakened by the repeated suggestion that nothing can be known with certainty.
This creates an opportunity for people who wish to avoid accountability. If every inconvenient image, voice recording or video can be labelled fake, the existence of deepfakes becomes a defence against reality itself.
The answer cannot be permanent suspicion.
A society in which people trust every image is vulnerable to manipulation. A society in which people trust no evidence is vulnerable in a different way. Journalism, justice, public institutions and ordinary relationships all depend on the possibility of establishing what happened.
The goal must therefore be better verification, not universal disbelief.
That includes examining where material came from, how it was obtained, whether independent evidence supports it and whether the source has been authenticated. Technical systems that record the origin and editing history of digital material may help. Labelling and detection tools may also play a role. But no single label, watermark or automated detector can carry the entire burden of trust.
Verification is strongest when technical evidence, accountable institutions and sound human processes reinforce one another.
Culture, Language and Unequal Exposure
AI-enabled fraud will not affect every person or community in the same way.
Language has historically limited some forms of mass fraud. Poor grammar, unnatural phrasing or unfamiliar cultural references could reveal that a message was not what it claimed to be. Generative tools reduce that barrier. They can produce fluent communication across languages and adapt messages to local contexts.
This may make deception more accessible to communities that have previously received fewer targeted messages. It may also allow scammers to imitate culturally familiar forms of authority: a family elder, religious leader, government official, employer or community organisation.
The meaning of an instruction is shaped by culture and power. In some settings, questioning an older relative or senior leader may feel disrespectful. In others, a request for family support may carry strong moral weight. Migrants may fear contact with authorities, while people navigating an unfamiliar system may depend heavily on apparent intermediaries.
These realities should inform prevention, but they must not become stereotypes. Culture does not make a person gullible. It shapes the relationships, obligations and signals through which trust is expressed.
Protection must therefore be accessible and culturally informed. Public information should be available in relevant languages and formats. Reporting routes should be clear and safe. Advice should recognise different family structures, communication habits and experiences of authority.
A warning that people cannot understand, access or apply is not an effective safeguard.
Verification Must Become a Habit
The strongest response to AI-enabled impersonation is a simple principle: an unexpected high-risk request should be confirmed through a channel the requester did not choose.
If a relative calls asking for money, end the call and contact them using a number already stored. If a colleague requests a payment, use the organisation’s trusted directory rather than replying through the incoming message. If bank details change, confirm the change with a known contact using an established process.
Families may also agree on a private verification phrase or question for emergencies. It should not be something easily discovered through public information, and it should form part of a wider checking process rather than becoming the only safeguard.
Several habits are especially useful:
Pause when a request combines urgency, secrecy and consequence.
Treat a familiar voice or face as one signal, not final proof.
Contact the person or organisation independently using trusted details.
Do not transfer money, disclose credentials or grant access solely because an incoming call appears genuine.
Ask questions that interrupt the script and create time to think.
Report suspected impersonation quickly, even if no money was lost.
These steps are deliberately uncomplicated. A defence that requires specialist analysis at the moment of crisis will fail many of the people it is supposed to protect.
Organisations Need Processes That Survive Impersonation
Awareness training alone is not enough. An employee may understand deepfakes perfectly and still authorise a fraudulent payment if the organisation rewards speed, discourages challenge or allows senior staff to bypass controls.
Organisations should design procedures on the assumption that email addresses, telephone numbers, voices and video appearances can all be imitated.
High-risk actions should require independent confirmation. Payments above an appropriate threshold, requests to change bank details, disclosure of sensitive information and changes to system access should not depend on a single communication or individual.
Useful controls include:
dual approval for significant payments and account changes;
call-back procedures using verified internal contact information;
clear rules for urgent and out-of-hours requests;
additional checks when payment details or communication channels change;
rehearsals that include voice, video and messaging impersonation;
fast internal reporting without blame or embarrassment;
consistent procedures for executives, employees, contractors and suppliers.
The final point is crucial. A control that disappears when a senior person appears to make the request is not a control. Scammers deliberately borrow authority because they expect hierarchy to silence doubt.
Leaders must demonstrate that verification is welcome. An executive who responds defensively to a legitimate security check teaches employees to comply the next time. One who thanks them teaches the organisation to resist manipulation.
Responsibility Cannot Rest Only With the Target
Public advice often focuses on what individuals should notice or avoid. Personal caution matters, but it cannot solve an industrial problem on its own.
Technology companies, financial institutions, telecommunications providers, employers, regulators and public bodies all influence the conditions in which fraud succeeds or fails.
Platforms can make impersonation easier to report and act more quickly when identities are misused. Financial systems can introduce friction around unusual transfers and changes of account details. Telecommunications providers can strengthen protections against spoofed calls. Organisations can limit unnecessary exposure of sensitive internal information while preserving legitimate transparency. Public agencies can provide accessible guidance and reliable reporting routes.
Media provenance systems may help people understand where content originated and whether it has been altered. Detection tools may support investigations. Neither should be presented as a permanent technical cure. Criminals adapt, systems make mistakes and authentic material does not always carry a complete digital history.
Resilience depends on layers: responsible technology, effective regulation, organisational controls, public education and human judgement.
Trust After Deepfakes
The age of synthetic media does not mark the end of trust. It marks the end of treating appearance as sufficient proof.
We will continue to depend on voices, images and digital communication. We will continue to form relationships online and make decisions at speed. The challenge is to ensure that consequential requests are supported by something more durable than recognition alone.
That requires a change in habit.
Instead of asking only whether a message looks authentic, we ask whether its source can be confirmed. Instead of treating verification as an accusation, we treat it as care. Instead of expecting individuals to identify every technical deception, we build systems that remain safe when deception looks convincing.
Scammers benefit when verification feels awkward, slow or disrespectful. Resilient cultures make it ordinary.
The answer is not to trust nothing. It is to trust through verifiable relationships and processes.
About This Collection
Scam, Trust and the Business of Deception examines the psychological, technological, cultural and commercial systems behind contemporary fraud.
The series accompanies the Cultural Intelligence Studio video collection Manipulated Trust and podcast collection Inside the Scam: Trust, Manipulation and the New Economy of Fraud.