Most voice deepfake detectors are tested on clean clips: a few seconds of synthetic speech, no background, no phone line. Attackers do not call like that. They call over a mobile network, they pause while you talk, and the better ones play a busy office behind their voice because it sounds real.
We run both sides of this. Our red team makes cloned-voice calls to companies that have asked to be tested. Our detector listens for synthetic voices on those companies' lines. That gives us something unusual: the same call, scored from the attacker's side and from the defender's side, hundreds of times.
Here is what we found when we scored a cloned voice calling a real phone number.
With no background sound, the detector catches the clone. It scored between 0.92 and 0.98 on a scale where 1.0 means "certainly synthetic". Comfortable.
Add an ordinary office background under the same voice, and the score fell to between 0.01 and 0.05. The caller was now "real". Same voice, same phone line, same words.
The reason is not that the model cannot hear the synthetic voice through the noise. It is how detectors add up a call. A typical detector cuts the audio into short windows, scores each one, and averages the scores. In a phone conversation the caller is silent about half the time, listening. Those silent windows contain only the office background, which is a real recording of a real room, so the model correctly calls them "real". Averaged in, they bury the synthetic windows. Office noise does not fool the model. It fills the gaps with convincing reality.
The proof is simple. Remove the stretches where the caller is not speaking and score only the speech. The same office-background call went from 0.015 to 0.88. The clean call went from 0.82 to 0.999. Nothing about the model changed.
The averaging rule we used is the standard one from the academic benchmarks that most commercial detectors are built on. We expect many products share this weakness.
This matters for two reasons.
First, it is cheap to exploit. The exploit is a background noise track, which every attacker already has.
Second, it is cheap to fix. Score only the speech, and treat "is there any synthetic speech on this line" as the question, rather than "is the average window synthetic". We shipped that change to our own detector this week. On live calls the attacker with office noise now scores between 0.6 and 0.99 instead of near zero, and a second synthetic voice on the same calls that used to pass as real now scores 0.98. The remaining gap, when the babble sits directly under the words, is a training problem and we are working on it.
Our red-team personas now offer the office background as an explicit setting, labelled for what it is: more convincing to people, harder for detectors. Our customers see both results on the same call: did the employee give something away, and would the detector have blocked the caller.
If you run a voice detector, test it with background noise over a phone line, with the caller silent half the time. If you train people to spot vishing, know that the realistic calls are the ones with a busy office behind them.
If you would like us to score a recording of yours both ways, write to us.