Why word timings need a human check
Speech models predict when each word starts and ends, and real speech pushes back. Where timestamps drift and how to review them before you export.
Timestamps are predictions, not measurements
Speech recognition models such as Whisper, OpenAI's open-source transcription model, do not measure time with a stopwatch. They generate text with timestamps as part of the same prediction, so a word's start and end are estimates shaped by the model's training.
Real speech resists tidy boundaries. Music under the voice, two people overlapping, a drawn-out vowel, or a switch between languages can all shift an estimated boundary away from where the sound actually sits.
Reviewing timings in Clipy
Clipy currently transcribes with Deepgram rather than Whisper, but the review habit is the same. after transcription, play the video from each subtitle and read against the audio, then nudge start and end times where a line appears late or lingers too long.
- Generate the transcript from your audio or video.
- Play each subtitle from its start time and listen to the words it claims to cover.
- Trim lines that start before the speech or hang over silence.
- Split a long subtitle if it holds two sentences with a pause between them.
When to trust and when to fix
Clean single-speaker speech usually lands close enough that small nudges finish the job. Music videos, interviews and code-switched speech deserve a full pass. The preview during editing is approximate, so play the exported file before publishing.
Common questions
Does Clipy use Whisper?
No. Clipy currently sends extracted audio to Deepgram for transcription. Whisper is described here because it is a well-known model, and the same timing caveats apply to any speech recognizer.
Why do timings drift more on songs?
Music masks word boundaries and models trained mostly on speech estimate them less confidently. Expect to adjust sung lines by hand more than spoken ones.
Further reading
Try the workflow
Prepare your own file and review the result before using it.
Open the related Clipy tool