Deepfakes, Chatbots, and AI Scams: What Every Business Needs to Know

‍

An employee gets a video call from the CFO, asking for an urgent wire transfer. The voice is right. The face is right. The request sounds exactly like something the CFO would say under deadline pressure. None of it is real — and by the time anyone checks, the money is gone. This isn't a hypothetical: a version of it cost one global engineering firm $25.6 million in 2024. Deepfakes and AI-driven scams have moved from novelty to a standard tool in the fraud playbook, and the businesses getting hit hardest are the ones still relying on the old tells — bad grammar, a suspicious link, an unfamiliar sender — that these attacks no longer have.

Key Takeaways

• AI-enabled fraud is no longer a future risk. The FBI's IC3 recorded 22,364 complaints with a reported AI nexus in 2025, totaling $893 million in losses, inside a broader $20.9 billion year for internet crime.
• Business email compromise remains the costliest social-engineering category by a wide margin — $3.05 billion in reported 2025 losses — and AI-generated voice and text are now folded directly into BEC schemes, not layered on top of them.
• Deepfake scams exploit familiarity, not carelessness. The traditional advice — watch for typos, hover over links — doesn't apply to a synthetic voice that sounds exactly like your CFO.
• Detecting a deepfake technically and verifying that a request is genuine are two different problems. Most protection advice addresses only the first one.
• We verify the person and the channel independently of the content itself — through open-source and public-record confirmation, not through media-forensics tools alone.

The New Face of Social Engineering

Social engineering used to rely on volume and probability: send enough phishing emails, and someone eventually clicks. AI-generated deception works differently — it relies on familiarity. A cloned voice, a synthetic video, or a chatbot fluent enough to hold a real conversation doesn't need to fool everyone. It needs to fool one person, for one moment, into treating a fabricated request as routine.

Three distinct attack types now fall under this umbrella. Deepfakes are synthetic audio or video — a cloned voice on a phone call, a fabricated video on a conference call — built from source material that's often publicly available: earnings calls, conference talks, social media clips. Chatbot scams use conversational AI to impersonate a real person or a legitimate service, maintaining a plausible back-and-forth rather than a single scripted message. AI-generated phishing produces messages that read as genuinely, individually written — no awkward phrasing, no generic greeting, often referencing real, specific details about the target's role or recent activity.

Why this matters: every one of these techniques is designed to defeat the exact heuristics most security-awareness training still teaches. "Check for spelling errors" and "don't trust generic greetings" are advice for a threat model that AI-generated attacks have already moved past.

How These Scams Actually Work

The attacks generally follow a consistent four-stage pattern, and understanding the sequence matters more than memorizing the technology behind any one stage.

Reconnaissance. Attackers gather source material from what's already public — executive interviews, earnings calls, conference keynotes, social media video, company org charts, vendor relationships mentioned in press releases or case studies. None of this requires a breach; most of it is willingly published.

Impersonation. That source material becomes training data for a cloned voice or a synthetic likeness — increasingly with a few seconds of audio or a handful of public photos, not the minutes of footage earlier tools required.

Engagement. First contact is calibrated to feel routine: a call that references a real project, a message sent at a time the target would plausibly expect it, a chatbot that responds naturally enough to hold a real exchange rather than a single scripted line.

Execution. The request itself — a wire transfer, a credential, a change to banking details — arrives wrapped in urgency and, often, a stated reason the normal process should be skipped just this once. The urgency is doing real work here: it's the mechanism that discourages the target from pausing to verify, framed as a cost of delay (a missed deadline, a lost deal, an angry client) rather than as a request that deserves scrutiny.

Why this matters: smaller and mid-sized businesses are disproportionately targeted at exactly this last stage, precisely because they tend to have less formal, less multi-step verification for high-value or unusual requests than larger organizations with dedicated finance-control teams.

The Business Impact You Can't Ignore

The direct financial exposure is the easiest part to quantify, and the numbers involved are no longer rounding errors. The FBI's Internet Crime Complaint Center recorded $20.877 billion in total reported cybercrime losses in 2025, with business email compromise alone — 24,768 complaints — accounting for $3.05 billion of that, the second-largest loss category after investment fraud. Separately, IC3 tracked 22,364 complaints carrying a reported AI nexus, totaling $893 million in losses, including voice-cloning-enabled distress scams with victims claiming losses over $5 million in 2025 alone.

Reputational damage runs alongside the direct loss. A fabricated video or cloned executive voice circulating even briefly — inside or outside the company — creates a credibility problem that outlasts the immediate fraud attempt, and is far harder to correct than a financial loss is to recover.

Operational disruption follows any incident serious enough to trigger a real response: the internal investigation, the process freeze on the payment types involved, the time diverted from normal work to figure out exactly what happened and what else might be exposed.

And regulatory exposure has grown alongside the threat. Depending on sector, a successful AI-enabled fraud incident can trigger obligations under frameworks like HIPAA, CMMC, or the FTC Safeguards Rule — turning what starts as a fraud loss into a compliance investigation as well. A healthcare provider whose staff is deceived into releasing patient data through a fabricated voice call, for instance, faces the same breach-notification obligations it would if the data had been taken through a technical intrusion — the regulator doesn't distinguish between the two causes when the same protected information ends up exposed.

Why this matters: the Arup case is instructive precisely because nothing about it required a technical breach. An employee at the firm's Hong Kong office joined a video call with what appeared to be the company's CFO and several other colleagues, and transferred $25.6 million across 15 transactions over the course of a week before the fraud was caught. Every participant on that call was fabricated. No firewall or endpoint tool was ever in play — the entire attack lived in the space between "this looks and sounds right" and "this is actually who it claims to be."

Where Verification Actually Fits

Most guidance on this topic stops at awareness: train staff to recognize the threat, tighten internal processes, add multi-factor authentication. All of that is necessary. None of it answers the harder question — in the moment a specific call, video, or message arrives, how do you actually confirm it's genuine, rather than plausible?

This is where the problem shifts from a security-awareness question to an investigative one. A synthetic voice or video is built to pass a human's in-the-moment judgment — that's the entire point of the technology. Confirming it's fraudulent (or genuine) means checking something the deepfake itself can't fabricate: whether the calling number or account has any independent history tied to the person it claims to be, whether the request matches how that person and process actually operate, whether the same request reaches you through a second, independently-established channel and gets the same answer.

We treat a high-stakes request — a wire transfer, a credential reset, a change to payment instructions — the same way we'd treat an unverified claim in a due diligence engagement: as something to confirm against independent, open-source, and procedural evidence, not something to accept because it sounded right. A caller-ID label or a messenger avatar is a weak lead on its own; tracing who a phone number actually belongs to takes independent cross-checking, not a single database entry. The same logic applies to a lookalike domain or a fabricated executive profile sitting behind a suspicious email — the OSINT methods used to expose that kind of impersonation infrastructure are the same ones that catch a deepfake — and to a social profile claiming to belong to a specific executive or vendor contact — the same process used to find and verify a real person's social media presence applies just as directly to confirming who's actually behind a profile a scam is built on.

How Molfar Verifies a Suspicious Request in Practice

A pattern we see repeatedly, anonymized from the kind of engagement that follows a near-miss or a live incident: a finance team receives an urgent request — by voice, video, or written message — to change payment details or release funds outside the normal process, with a plausible reason given for the urgency and for bypassing the usual sign-off.

Rather than evaluating the request in isolation, our approach starts with the identity behind it. We check whether the calling number, email domain, or account has any independent public history consistent with the person or organization it claims to represent — a domain registration date that doesn't match a company's actual founding, a phone number with no prior association to the claimed identity, a professional profile created within days of the request rather than years before it. We then establish a second, independently-sourced channel to the actual person or organization — not a number or address supplied in the suspicious communication itself — and confirm the request directly. Where the request references a real deal, project, or relationship, we verify that detail against its own independent record, rather than accepting it as proof because it was accurate.

What breaks a well-executed deepfake or AI-phishing attempt is rarely a more sophisticated detection tool. It's the discipline of confirming identity and channel through a path the attacker doesn't control — which is exactly the same discipline that underpins verifying a vendor, a counterparty, or a beneficial owner in any due diligence engagement. The content of the message was never the reliable part. The verification path around it is.

FAQ

Can deepfake detection software solve this problem on its own?

No. Detection tools analyze media for technical artifacts — inconsistent lighting, unnatural blinking, audio-visual sync issues — and they're a useful signal, especially as generation quality improves faster than most detection tools can track. But they answer "does this look synthetic," not "is this actually the person it claims to be." A request can pass every technical check and still come from someone who isn't who they claim to be, if the underlying identity was never independently confirmed.

What's the single most effective protection against a deepfake-based scam?

An independently-established second channel for any high-stakes request. If a call, video, or message asks for a wire transfer, a credential, or a change to payment details, confirm it through a phone number, email address, or in-person contact that was established before the request arrived — never through contact information the request itself supplies.

Are small and mid-sized businesses actually more at risk than large enterprises?

In relative terms, yes. Larger organizations tend to have more formal, multi-step approval processes for high-value transactions by default, which creates more opportunities for a fraudulent request to be caught. Smaller businesses often route the same decisions through fewer people with less friction — which is precisely what makes the "urgent request from a senior figure" pattern effective against them.

Does multi-factor authentication protect against these scams?

It helps against a specific subset — account takeover and credential-based access — but it doesn't address a scam where the attacker never needs to access a system at all, only to convince a person to act. A deepfake CFO asking an employee to authorize a wire transfer doesn't need to defeat MFA; it needs to defeat the human decision to comply.

How is this different from traditional phishing?

Traditional phishing relies on volume and recognizable tells — generic greetings, poor grammar, suspicious links — that trained employees learn to spot. AI-generated scams are personalized, contextually accurate, and often delivered through a channel (a voice call, a video call) that doesn't carry the same visual tells as a text-based email. The defense has to shift from pattern recognition to identity and channel verification.

What should a business do immediately after suspecting it's been targeted?

Freeze the specific transaction or process involved before doing anything else, then verify independently rather than re-contacting whoever initiated the request. Document the exact channel, timing, and content of the request for the ensuing investigation, and treat the incident as a verification failure to fix in the process — not just an isolated bad actor to report.

‍