A familiar voice on the phone is no longer reliable proof of identity. A synthetic recording can sound faithful, while a detector that succeeds on laboratory files fails after the signal has passed through a particular telephone codec. Defence therefore has to work with the payment process and a second verification channel, not with hearing alone.
What we know: A synthetic voice can sound convincing, and the voice channel itself does not confirm the caller’s identity. The risk can be reduced by an agreed procedure, a callback to a known number, and a second approval of the payment.
What we do not know: There is no universal sample length, and no detector reliability that holds for every model, language and telephone channel. From the sound alone, without more evidence, one also cannot determine the legal qualification of a particular case.
1. A voice that is no longer a seal of identity
Picture the Monday meeting after an incident. The director wants a new detector, because “AI can do everything now”. IT proposes a ban on voice messages. Finance insists on a second channel for every payment above a set amount. The lawyer cites the AI Act and the Criminal Code. Communications wants training. The assistant who sent the money wants to know how she was supposed to recognise a voice that sounded exactly like the boss.
At first glance this is a dispute about technology. In fact each person is describing a different problem. The director compares hearing and the model. IT describes the attack surface. Finance describes whether the payment can be reversed. The lawyer describes penalties after the act. Communications describes habit. The assistant describes the cost of a mistake in ten seconds.
A detector can label the sound correctly and still fail to stop the money from being sent. A security design therefore separates the channel, the amount threshold, a password agreed in advance, approval by a second person, an audit trail, and the ability to stop the payment.
A comparison of vocoders is only a partial test. For a real fraud one has to evaluate, separately, the quality of the sound, human recognition, technical detection, identity verification, and the approval rule for the payment.
A voice is not an identity. Fidelity is not origin. A detector is not a lock. And the law after the money has been sent is not a defence.
— Jiný Kontext
2. What a fraudulent call is made of
When a familiar voice comes out of the telephone, it is easy to form the impression that “he is the one calling”. But the result usually arises in a chain. Each layer can raise the quality of the attack, lower it, or change the type of error. Two frauds that use the same clone therefore need not end the same way, and the same clone can behave differently according to the channel, the pressure, and whether a second step exists.
Five layers of one emergency call
- Raw material: a public video, a voice message, a podcast, a “sorry, wrong number”.
- Synthesis: a model that builds a new sentence from a short sample.
- Channel: a call, a voice message, a video conference, a robocall.
- Script: distress, authority, time pressure, a non-standard account.
- Control: a password, a callback, a payment threshold, a rollback.
The base model determines an important part of the fidelity, but it is not the only actor. An attacker can insert an SMS before the call, send an account number after the call, cut off the chance to ask a question, or wait until the victim is alone. The same voice in a calm conversation and in “five minutes until they detain him” is not the same exercise.
This is where the first error of the public debate is born. A headline announces a contest between the ear and AI, but in fact it compares two different chains: laboratory listening with a warning, and a telephone in the kitchen without one. Or, conversely, it measures that a detector hit a dataset and then presents the result as a verdict on a family payment. An honest comparison has to state not only the name of the model, but also the channel, the codec, the presence of pressure, the time available, and the method of control.
Without a cut through the layers, the call becomes magic. That cut determines where to turn when something fails. If the voice sounded right but the account number was new, the channel failed. If no password existed, administration failed. If the detector stayed silent on GSM, the transmission layer failed. The sentence “AI deceived us” is the beginning of a diagnosis, not its end.
Illustrative scheme: The same clone can, in another channel, under another pressure and with another payment threshold, produce a different harm.
3. The best defence is the one whose error you can bear
In detection, what matters is not only the success rate, but also the consequence of a false alarm and of a forgery that was missed. A wrongly flagged family recording has a different impact from a transfer to a fraudster’s account. The process therefore has to be stricter where the decision is hard to reverse.
In its generative-AI risk profile, NIST points to erroneous content that is presented convincingly, and to the fact that risk does not grow only with the frequency of error. It also grows when the form tempts disproportionate trust.[17]
| Type of failure | What it looks like | How to test it | What limits the harm |
|---|---|---|---|
| Faithful clone | The voice fits; the sentence is new | A practice call with an agreed password | A second channel; a ban on paying on voice alone |
| Pressure | “Right now, otherwise…” | Whether the protocol can say no | A time lock; an amount threshold |
| Channel | An account in an SMS, a new contact, a video call | Whether the number is in the directory | A callback to a stored number |
| False detection | The detector stays silent, or else shouts all the time | A dataset versus a real codec | A process, not a single score |
That also changes the question we are asking. We are not asking only which system will recognise a forgery on average. We are asking which one will fail in a way that our process can catch. A family can tolerate ten false suspicions, because it can bear one delayed call. An accounts department cannot tolerate a single hidden change of account number. A developer can accept a false alarm if a payment test fails. A mother on the telephone must not mistake fluency of voice for identity.
Human listening is only one control. It also matters to record the request, to limit payment authority, and to require a second approval. Such a process can be more resilient than even the laboratory’s best detector on its own.
4. What laboratory 73 percent means
Detection tests are needed, because they replace part of impression with a repeatable trial. The problem begins at the moment when one score starts to stand for a person’s entire ability.
In 2023, in PLOS ONE, Mai, Bray, Davies and Griffin presented genuine and synthetic recordings to 529 listeners in English and Mandarin and asked them to mark the fakes. Listeners identified the deepfake in 73 percent of cases. There was no difference between the languages. A short familiarisation with examples improved the result only slightly. This is a laboratory task in which people knew they were looking for fakes and listened to samples one sentence long. It is not evidence of behaviour on an unexpected telephone call. It does show, however, that even under favourable conditions hearing is not enough for a reliable defence.[6]
Every test contains hidden decisions. Did the listeners know it was a test? Was it a telephone, or a clean recording? Which codec? Which model? How many seconds? Is identity being judged, or only whether “it sounds artificial”? An answer to any of these questions can change the number. Newer commercial clones may be harder; older synthesis easier. The authors themselves note that the algorithms used were already not the latest at the time.
Five facts without which 73 percent is only an advertising number
- The exact synthesizer version and the language of the samples.
- Whether people were warned that deepfakes would be present.
- The length of the recording, the channel and the compression.
- Whether naturalness is being judged, or a match of identity.
- The sample size and the interval of uncertainty.
For buying a defence, a small internal test is therefore more important than someone else’s overall ranking. Thirty real company scenarios — with a real payment threshold, the Czech language, a telephone codec and real time pressure — can say more than a thousand laboratory sentences. A public test helps to select candidates. The decision has to be made by the protocol.
Telephone networks cut the band and compress the signal, which can weaken some detection marks, reshape others, and at the same time add new artefacts. Without a test of the particular codec one cannot claim that compression of a track generally “erases” it. Listening on a call is therefore not the same task as listening to a clean file in a study.
5. Compare channels, not vocoders
The diagram splits the attack by voice source, channel, time pressure and the target decision. Defenders can then test a particular point of failure instead of a general ranking of generators.
| Channel | Main operating stance | What to check in the exercise | Typical reason the attacker chooses it |
|---|---|---|---|
| Voice message | One-way pressure; no chance to ask back | Whether the protocol forbids paying from a voice message | Live synthesis is not needed |
| Telephone call | A familiar number, or spoofing | A callback to a stored number | Distress and interruption |
| Video call | The authority of a group; several faces | A second channel, a password, a delay of the payment | A company transfer |
| Robocall | Scale; political or commercial content | The origin of the number, a complaint, an archive | Thousands of handsets at once |
| Family emergency | Emotion, haste, bail | A family password; a ban on an instant transfer | The victim already believes the voice |
The rows are not a judgment on the quality of the models. The same tool offers several modes, and each can change within a few months. The map is meant to prevent another error: that an organisation buys an excellent detector into a process that cannot see the payment, cannot stop it, or locks the decision into hearing from which it is later hard to walk away.
How to read the scheme: Neither the position nor the size of the elements expresses a score. The diagram separates an audio forgery, a verification of identity, and an irreversible payment decision.
6. Five seconds of recording: raw material, not a crime
Cloning a voice used to require hours of studio recordings. In a survey of the misuse of artificial intelligence, Europol describes tools that, according to their advertising, need only a five-second recording of the target.[1] This is a warning from a security agency and a description of tools that are available, not a laboratory measurement of every model in use today. It does, however, show a threshold: the raw material is short, and often public.
A sample can be taken from a voice message, from a video on a network, from a conference, from a podcast, or from a “sorry, wrong number” call. That is not magic. It is the same logic as a password written on a slip of paper: whoever has the raw material has a start.
Voice synthesis also has legitimate uses — a voice for patients after loss of speech, accessibility, dubbing with consent. It does not follow that every clone is a crime. It follows that raw material and the act are two layers. A public video is not consent to an emergency message. Five seconds is not fraud. Fraud begins where fidelity is joined to a request that cannot be undone.
An illustration, not a report. An attacker downloads ten seconds from a public podcast and makes a voice message for the assistant: a deposit of 180 000 CZK, a notary, an account in an SMS, urgent. The colour fits; a café is in the background. Process control — reverse verification through another channel — is missing. The sound was faithful. Layer five failed.
7. Pressure and channel: the fraud does not begin with the model
The success of a fraud depends on psychological manipulation and on time pressure as much as on the fidelity of the AI. That is reasonable to read as a diagnosis, not as an excuse. A model without a script is a demo. A script without a model was already vishing before synthesis.
In its report on the state of cybersecurity of the Czech Republic for 2023, NÚKIB ranks phishing and its derivatives — including vishing — among the most common techniques. On vishing it states expressly that the numbers of real organisations, the names of officials, or a voice with the help of AI can be misused.[10] This is a description of threats inside the authority’s incident framework, not an audit of every Czech telephone call.
In a campaign on fraudulent telephone calls, the Czech Banking Association stated that for 2023 it recorded almost 70 thousand attacked clients and average damage of more than 19 thousand crowns, with a year-on-year roughly threefold rise in the number of attacks. In the same campaign the Police of the Czech Republic put the figure at almost 200 million CZK lured from accounts, on average more than half a million crowns a day; the highest recorded amount from one victim in Prague was almost 5.5 million CZK.[18] These numbers describe telephone fraud as a whole, not the share that already used a deepfake. They are a map of harm. They are not a verdict on a vocoder.
A process that forbids making a payment on a voice request alone can reduce the risk sharply even against a convincing clone. It is not absolute protection: an attacker can hit the second channel, obtain the password, or abuse an internal exception. Hearing by itself is therefore a weaker control than several independent steps.
Three tests of an urgent call
The first is a test of time: can the request be postponed by twenty minutes? The second is a test of origin: am I calling back the number I have stored, not the number from the call? The third is a test of contradiction: does the password fit, the place, the dog’s name, yesterday’s detail that a clone from a video does not have? A product that “sounds a second more faithful” need not be more dangerous if the protocol forces these three tests.
8. The harm is empirical. It is not science fiction
The effects are no longer only laboratory effects. Before the New Hampshire primary in January 2024, thousands of voters received a robocall with a synthetic voice of Joe Biden that urged them not to take part. The Federal Communications Commission (FCC) imposed a civil penalty of 6 million dollars on the organiser.[8] A criminal jury in New Hampshire later acquitted the defendant. That is an important distinction: a civil sanction for spoofing and an AI voice is not the same as a criminal verdict on the suppression of voters. The case shows the scalability of the channel, not that every political call is a deepfake.
In January 2024 an employee of the Hong Kong office of Arup, according to reports of the case, authorised transfers in the order of 25 million dollars after a video conference in which synthetic avatars of management appeared. The fraud was revealed only by a check against headquarters, after the money had already gone.[9] This is a reported company incident, not a public court file in this text. The mechanism is still readable: the authority of a group, a live image, several transfers, a check that came too late.
The Czech harm from vishing of the 200 million CZK mentioned above for 2023 is an order of magnitude elsewhere and on another channel. A comparison of the USA, Hong Kong and the Czech Republic is therefore not one ranking. It is a map showing that distress, authority and scale work in different fields. The definition of “deepfake” in police statistics need not match the definition in a laboratory paper. When the definition fits badly, it is more honest to say so than to add pears together.
9. A detector is a race. It is not a lock
Against a synthetic voice there are tools that look for spectral traces, irregularities in breath, fingerprints of generators. Projects such as vera.ai and work on the frequency analysis of synthetic media describe this line.[2][4] Pindrop published a forensic view of the New Hampshire robocall.[3] On test sets detectors can score high. That is a property of the dataset, the version of the attack and the threshold. It is not a lock on the handset.
Attackers can adapt models and post-processing to known detection marks. Telephone compression can worsen some marks or change them. A false-positive alarm burdens operations, and a false-negative stays silent. The concrete extent of this effect, however, has to be shown by a study or a test of the given detector, generator and channel.
The Coalition for Content Provenance and Authenticity (C2PA) makes a different bet: not “can you tell that it is fake”, but “can you prove where it is from”. Content Credentials are meant to carry origin, not impression.[5] That does not move the problem into the sky. It moves it to a place where it can be tested: does the file have a signature? Was it broken? Who is the issuer?
The largest competitive advantage of a defence therefore need not be a model. It can be order in payments. And the largest weakness need not be a hallucination of the ear. It can be the genuine voice of the boss, who nevertheless never sends an account in an SMS.
10. The law punishes ex post. The money leaves ex ante
The duties in Article 50 of the AI Act apply, according to the timetable of the regulation, from 2 August 2026 and distinguish two roles. Under paragraph 2, a provider of a system that generates synthetic content has to ensure machine-readable marking and the detectability of the output to the extent that is technically feasible and with the exceptions stated. Under paragraph 4, a deployer has to disclose that image, audio or video content constituting a deepfake was artificially created or manipulated; the regulation sets exceptions and a limited form of disclosure, among other things for some artistic, satirical or fictional works. The duty does not verify the identity of the caller, and an offender can breach it.[12][21]
A voice recording is personal data if it relates to an identified or identifiable person. It becomes a special category of biometric data when it is technically processed for the purpose of unique identification. The legal basis and further duties in training or cloning depend on the purpose, the role, the source and the context; they cannot, without the facts of the case, be declared an automatic breach.[13]
The Criminal Code of the Czech Republic has no separate offence of “creating a deepfake”. According to the particular conduct, the consequence and the fault, misuse can fulfil, for example, the elements of fraud (Section 209), damage to the rights of another (Section 181) or defamation (Section 184); the legal qualification is neither automatic nor exhaustive.[14] A criminal penalty, moreover, does not by itself return a payment that has already been sent.
It does not follow that the law is useless. It follows that it is a different layer from the payment protocol. Whoever melts these two layers into one sentence, “it will be banned”, buys a feeling. They do not buy reversibility.
11. What labelling of synthetic content can do
Before a watermark, a person had to check whether the voice sounded right. With a label one checks something more complicated: whether the mark exists, whether it belongs to this file, whether it was stripped by compression, and whether it applies at all to a telephone call, which rarely carries a mark. A label is navigation toward origin. It is not automatic origin.
One message can combine three true traces into a false act. The first is a faithful voice. The second is the real name of the boss. The third is real pressure of a deadline. Each part works. The synthesis is the fraud. At other times the system uses a high-quality clone on a channel that would have stopped even a bad clone: a new account and the word “immediately”.
A four-step check of a call
- Existence: is this a live call, or a recording?
- Authority: is the number from my directory?
- Support: does the request match the agreed process?
- Scope: do the amount, the account, the time and the exception fit the rule?
The most reliable procedure separates hearing from sending. First a verification through a second channel is created. Only then the payment. If the password does not fit, or the number is not stored, a claim is not promoted to an order merely because the voice sounded right.
A security process is meant to shorten the path to verification, not to pass the sound off as evidence. For a harmless family message, hearing may be enough; for a transfer, the last step has to be governed by a rule that a familiar voice cannot bypass.
12. Origin is not hearing, and hearing is not identity
The large word “authenticity” is often translated into the sentence “I heard it with my own ears”. Technically a system can accept a sound that convinces the ear. It does not follow that the sound carries an identity. C2PA documentation separates content from credentials: a signature about origin, not an impression about the colour of the voice.[5]
Studies of human detection show that hearing is a map in one task. It is not a verdict in another. Mai et al. measured the marking of a fake in the laboratory.[6] A company transfer measures something else: the willingness to break a process under authority. A family bail measures feeling. Court evidence will measure a chain of origin, not how a witness remembers the call.
The memory of the product — a recording of the call, a switchboard log, a signature on a file — is another layer. It can help forensics. It also creates a risk of false certainty if the recording itself becomes “evidence” without carrying origin. A file can be generated. A log can exist. That still does not give an identity.
How to test an urgent request
Put into the exercise a familiar voice of a colleague with a knowingly wrong password. Ask about the contradiction between an account in an SMS and the internal list. Require a delay. Repeat the task in a voice message and on a call. Do not evaluate only whether people “recognised AI”. Record whether the protocol stopped the payment.
On a large corpus of calls, a well-designed rule and a smaller relevant exception is usually better than a mechanical “listen more carefully”. The whole company inside a detector is not a strategy of trust. It is often only a more expensive way to hide the decisive “call back” in the middle of a dashboard.
13. Voice privacy is not a switch. It is the whole flow
The question “may my voice be cloned?” is important, but too narrow. A person and a company also have to know where recordings of meetings are stored, for how long, who transcribes them, whether records are kept, which third parties process the voice, and whether a public webinar sends raw material into the world. “We do not use it for training” does not mean “nothing is on YouTube”.
The differences among consent to dubbing, a recording of a meeting and a public podcast are sharp in the legal texts.[13] It does not follow that every company call is suitable for publication. It follows that raw material cannot be judged by the logo of a platform without the name of the account, the licence and the reach.
The practical minimum is a map. It begins with the speaker, continues through the microphone, the cloud, the network, the clone, the call and the payment, and ends at the place where the payment can be stopped or the voice taken down from a public channel. Without this map, a claim about privacy is only a description of one part of the pipe.
14. The price of a recording is the price of raw material, not the price of fraud
The market uses several price languages at once. A cloning tool may cost a subscription. A detector another. Training a day. Average vishing damage according to ČBA of more than 19 thousand CZK per attacked client, a peak in one case in the order of millions, a company video call in the order of tens of millions of dollars.[9][18] A direct comparison of price lists without knowledge of the payment threshold is therefore misleading.
A cheap detector can be excellent as a filter of voice inboxes. The same detector can be expensive on a task that the victim hears once, under pressure, and that in the end requires weeks of recovery. Expensive dual approval can be pointless for sums of a hundred crowns. It can be cheaper on a one-off transfer with a high cost of error if it reduces the number of successful emergencies.
The real price of an accepted call
Channel + time + verification + the cost of a false alarm + the cost of a payment that was sent + recovery + loss of trust + future dependence on one detector.
A call is “cheap” only when it stretches the check and can really be used as a prompt, not as an order.
That is why it is useful to measure the price per stopped payment, not the price per detected clone. Stopped means that the result met criteria of the process given in advance, even if the ear was wrong. This metric can reverse a ranking that looked unambiguous from the laboratory 73 percent.
15. What to introduce in a family and in a company
The first exercise should rest on a concrete scenario, language, telephone channel and financial threshold. The table shows which control regimes to compare under the same conditions; it is not a general ranking of detectors.
| Need | First group of measures | Decisive test | Unacceptable shortcut |
|---|---|---|---|
| Family emergency | A password, a ban on bail by telephone, a callback | A practice call with no warning of the day | Relying on hearing |
| Company payment | A second channel, a threshold, a delay, a list of accounts | A practice “boss at the notary” | An exception for management |
| A video call with authority | Verification outside the call; more eyes on the payment | The Arup scenario in miniature | Trusting the face in the window |
| Political robocall | Do not spread it; verify at the primary source | Whether the appeal changes behaviour | Sharing “so that people know” |
| A public voice of management | Less raw material; C2PA where it is possible | Whether the webinar is easy to clone | A whole podcast without a limit |
| Forensics after the act | A recording, a log, origin, not the hearing of a witness | Whether the chain will stand | A judgment by impression |
Czech use needs a separate test. An English dataset will not measure whether a clone hit the intonation of “mami”, inflects correctly, or presses for bail in crowns. A test set should contain a voice message, a spoofed number, an SMS with an account, and a scenario in which the correct answer is “I will hang up and call back”.
Some teams need more than one layer. One for the family, a second for accounts, a third for public appearances. That is not a failure of standardisation. It is the same decision as using one lock on a bicycle and another on a safe. The condition is that the split is intelligible and that people know where the final authority to pay arises.
16. A fourteen-day protocol that will say more than a detector
A defence can be built without a six-month tender and without an impulsive purchase. A short, disciplined pilot is enough. Its aim is not to prove that a detector can do something impressive. The aim is to find situations in which hearing fails, and to see whether the process can stop the harm.
Fourteen days from impression to a rule
- Days 1–2: Write down 20 to 40 real payment and emergency scenarios, their frequency and the cost of error.
- Days 3–4: Prepare the same inputs, passwords, lists of accounts and a scoring sheet.
- Days 5–7: Play each important scenario at least three times. Keep a register: hearing, password, callback, payment.
- Days 8–10: Check limits at the bank, delay, logs, deletion of recordings and public raw material on the web.
- Days 11–12: Calculate the price of a stopped payment, including false alarms.
- Day 13: Stress: a voice message, a video call, “immediately”, a new account, a child, management.
- Day 14: Decide according to the matrix, document exceptions and set a date for the next review.
What needs to be tested
A channel test repeats the same request in a call, a voice message and a video, because each carries different traces. A threshold test examines whether the process contains an exception for management. A data test separates a public webinar, an internal recording and a live telephone call.
The pilot has conditions for elimination. For example: a single unannounced payment from a voice, acceptance of a new account without a second channel, an inability to stop an outgoing payment, or a detector that cannot be turned off when it shouts all the time. A high average score of the ear must not override an error whose impact the organisation cannot accept.
The last output is not the name of a detector. It is an operating rule: one does not pay from a voice, the password holds, a callback is mandatory, raw material is limited, and the decision will be opened again in a quarter of a year. Models change. Without a revision date, even a good protocol turns into a superstition.
17. Two similar calls can have a different regime
With a voice it is especially important not to blur the boundary among family, company and a public appearance. The same person can read a fairy tale on a network on Tuesday and approve invoices on Wednesday. Raw material from Tuesday serves Wednesday’s clone. Privacy and thresholds differ. The cost of error differs.
The difference is not cosmetic. The exercise has to record in which regime the test is run, what the threshold is, and who may release the payment. Otherwise the company tests hearing on a clean recording and by mistake presents it as a security property of Monday’s emergency SMS.
Before sending money after a voice
- Hang up. Call back the stored number.
- Require a password or a detail that is not on the video.
- Do not accept a new account from the same channel.
- Hold the delay even if the voice is in a hurry.
- In companies: a second person, a list of accounts, a ban on the exception “because it is the director”.
The largest advantage therefore need not be a detector. It can be the boredom of the process. And the largest weakness need not be synthesis. It can be a genuine voice that nevertheless never breaks its own rule — until the rule does not exist.
18. Loss of trust after a fraudulent call
The first generation of fraudulent calls sold a fake bank. The next sold a fake grandson. Today’s tools sell something larger: the fidelity of the voices of people we already have in the directory. That is why attackers do not compete only for the best vocoder. They compete for context. For public recordings, for the habit of paying from a telephone, for an exception for management, for the fact that until recently “hearing it” was still enough.
The more convenient the voice is as an order, the higher the price of leaving that habit. People stop distinguishing a relationship from the channel in which they heard the relationship.
That does not mean that voice synthesis is bad. It is precisely synthesis that gives a voice to those who have lost it, and that saves work where there is consent. It means that it should be used deliberately. A family and a company need a portable password, an emergency procedure that does not depend on hearing, a limit on raw material, and a regular test of whether the original reasons for trusting a call still hold.
The best system is therefore not the one that never errs. We do not have such hearing. Nor is it the system with the highest laboratory percentage, the best watermark, or the strictest law after the act. It is the system whose controls match the real payment and whose errors are visible, reversible, and economically and humanly bearable.
At the moment when a voice only says hello, we are choosing a channel. The moment it asks for money, a password, or silence about an election, we are choosing a structure of trust. And then the question is no longer: “Will I recognise a deepfake by hearing?”
What decides is whether we verify the request through a second channel before trust in a familiar voice turns into an irreversible payment.
— Jiný Kontext