GUIDE · METHOD

How we measure WR3's figures

By · Published · Updated

Every figure we publish about WR3 comes from real 2026 focus group recordings, compared with a reference made by people. The main one comes from a manual verification of the 6 sessions of a 2026 study: WR3 got the speaker right in 98 to 99% of turns, and in 76 to 86% of the passages where several people spoke at once. Separately, an earlier internal measurement on 520 turns from two sessions gave 98 to 99% on the doubtful stretches WR3 decides on its own: a different figure, on a different sample. Whatever WR3 does not assign on its own is reviewed by a person. Here we explain what each figure measures, on what sample, and what we have not measured yet.

Who is speaking: the manual verification of a 2026 study

How we checked it. A person hired for this transcribed the 6 sessions of a 2026 study by hand, on paper: about 2 hours each, 12 participants per session on average, about 67 in total. We then compared, turn by turn and listening to the recording, who said each line according to that transcript and according to WR3. WR3 got the speaker right in 98 to 99% of turns, and in 76 to 86% of the passages where several people spoke at once. It is our own measurement, not audited by a third party.

Manual verification of a 2026 studySampleAccuracy
Who was speaking, per turn6 sessions of a 2026 study, about 12 h, about 67 participants98 to 99%
Passages where several people spoke at onceThe same 6 sessions, with the same manual verification76 to 86%

This is the figure we show on the home page and in WR3 in figures. The one in the next section is a different, narrower measurement.

An earlier internal measurement: 98 to 99% on the doubtful stretches

The reference. In two blocks of a real 2026 focus group wave, a person labelled by hand who was speaking in a sample of the doubtful turns. That human review is the reference we measure against.

What we compare. For each turn WR3 assigns on its own, with nobody touching it, that the person also assigned, we compare who WR3 assigns it to and who the person assigned it to. We count whole turns: each one ends up with the right person or it does not. It is an internal measurement, made in October 2026.

SampleWR3 turns comparedWith the right personAccuracy
One block223218 of 22397.8%
Another block297294 of 29799.0%
Both sessions520512 of 52098 to 99% depending on the block

What it covers and what it does not. Most speech is assigned directly by each person's own microphone. The figure covers the doubtful stretches, where a person's own microphone is not enough to decide who is speaking: 1,443 turns (37.6 minutes) in one block and 1,270 (31.5 minutes) in another, in blocks of about two and a half hours. In those stretches WR3 decided 505 turns on its own, 21.6 of 37.6 minutes (57%), and 480, 19.8 of 31.5 minutes (63%): more than half of the doubtful time. The 223 and 297 above are the turns among them with a human assignment to compare against. A person reviews the rest of the doubtful time in WR3 before delivery, listening and deciding. The figure does not say how right the machine would be on that part, because it does not decide it.

The margin. These are two blocks of one wave, not an average of many waves. With samples this size, each turn moves a block's figure by 0.3 to 0.5 points. We will publish new measurements as we make them, whether they go up or down.

An older measurement. We used to measure differently: second-by-second agreement over 141 moments of one block, including the human review of flagged moments, gave 90.4%. The two cannot be compared: that one counted seconds of all the audio, this one counts the doubtful turns WR3 decides on its own.

What a "verified quote" means

A quote passes two checks before it reaches a report.

  1. Literal check, automatic. When the AI proposes a quote, the system checks that its words appear complete and in order in the transcript of a single turn, ignoring case, accents and punctuation; that the turn belongs to one of the people the idea is attributed to; and that it has at least four words. The text kept is the transcript's, never the one the AI wrote. If it does not match, the quote is rejected.
  2. Human verification, by listening. A person listens to every quote in the recording and marks it verified or rejected. A finding cannot be approved without at least one verified quote, the report does not publish while an approved finding lacks a verified main quote, and only verified quotes reach the report.

The first check makes sure the quote is in the transcript. The second, that the transcript says what was said in the room. In the real wave, 74 of 78 quotes were verified by listening; the rest did not go into the report.

Where "11 of 120 seconds" comes from

To measure transcription quality we took two-minute windows of a participant's lapel microphone. In one of them, the person wearing that microphone was the loudest voice in only 11 of the 120 seconds. In another window, in 37 of 120. The rest of the time, the microphone was picking up the conversation at the table.

That is why WR3 does not transcribe whole microphone files: it would transcribe the neighbors as if they were the wearer. It first finds each turn and its owner, levels the audio of each turn, and only then transcribes.

What we have not measured yet

Behind all these measurements there is one principle: the worst error is the silent one. When WR3 cannot decide who is speaking, it does not guess: it flags the moment and a person resolves it by listening. In the older measurement, where the speaker's microphone was not synchronized, the machine was wrong on 5 moments and flagged 4 of them for review, and after review no silent error was left. A flag beats a guess.

To see how each quote looks in a deliverable, browse the sample report, built from a fictional study.