EveryWord: subtitles that light up word by word as they are spoken
A thirsty Crow found a Pitcher with some
water in it, but so little was there that,
try as she might, she could not reach it
Every video already has subtitles. None of them have words. A subtitle file says what was said and when the line starts. It never says which word is being spoken, so nothing can highlight one. EveryWord adds the only thing missing, to the file a video already ships with.
Below is Sintel, distributed by the Blender Foundation with English subtitles inside the file. Those cues are the filmmakers'. EveryWord only gave them word timings, and you can read both files. Nothing here is a mock-up and nothing here was written by us.
This is the technique that ran on Indian national television for two decades, reached an estimated 200 million viewers, and has been national broadcast policy there since 2019. It worked because it rode programmes people had already chosen to watch. That only becomes possible for the rest of the world's video when the captions it already has can carry a word.
Measured, not promised: against gold-standard word alignments on LibriSpeech (27 speakers, 788 words we did not record), the highlight lands within 30 ms of the spoken word (median), never lights a word early beyond 150 ms in 766 matched words, and breaks lines at real pauses 42 percent of the time where timing-blind chunking manages 2. The same run on test-other, the split LibriSpeech itself calls hard, gives the same 30 ms and the same zero. Method and full results
What it is
A way to give any video's existing subtitle track word timings, so the word being spoken can be lit as it is spoken. Point it at a film and the captions it already has, and it returns the same words with a time on each. You just watched it work on a real film; the same renderer runs on the EveryWord Fire TV app.
Why it exists
130 million US adults read below a 6th grade level, and their children are far more likely to struggle too. In India, word-synced subtitles on ordinary television reached an estimated 200 million viewers over two decades, and a five-year study measured 32 points more children becoming good readers. It worked because nobody signed up for it: the reading practice rode programmes they had already chosen. Amazon proved the same mechanic for books with Immersion Reading. No television platform has shipped it for video, because until the captions can carry a word, there is nothing to light.
How it works
The subtitle file supplies the words. Amazon Transcribe supplies the timings. The recogniser is never asked what the words are, so one that hears “their” for “there” cannot put the wrong spelling in front of a child: zero word error by construction. The line breaks stay the subtitle author's, because a human chose them. Then the renderer lights the word being spoken and your eyes follow it whether you mean to or not. That is the whole trick, and it is measured: median 30 ms from voice to highlight, and not one word in 766 lit more than 150 ms early.
The evidence
EveryWord implements an intervention that other people have already measured at national scale. Every figure on this page is sourced below, and the full list with our own corrections is in the repository.
A five-year study of 13,000 people who could initially read little or nothing found 32 percentage points more children became good readers than in the unexposed group, and 9 points more adults. Newspaper reading rose from 33.7 to 70 percent. Cost: $0.004 per learner.
Kothari and Bandyopadhyay, catalogued by the UNESCO Institute for Lifelong LearningMore than 100 empirical studies find captions improve comprehension of, attention to, and memory for video, with people learning to read among those who gain most.
Gernsbacher, Policy Insights from the Behavioral and Brain Sciences, 201554 percent of US adults aged 16 to 74, about 130 million people, read below a sixth grade level. The estimated cost to the economy is $2.2 trillion a year.
Gallup for the Barbara Bush Foundation, 2020In 2023, 28 percent of US adults, 58.9 million people, scored at literacy Level 1 or below, up from 48 million in 2017. The problem is growing, not shrinking.
OECD PIAAC Cycle 2, US results
Full evidence file · Our measured timing accuracy · Open source (MIT)
Every video already has subtitles. None of them have words.
That is the gap, and it is why nobody has shipped this for video. A subtitle file says what was said and when the line starts. It never says which word is being spoken, so nothing can highlight one. EveryWord takes the file a video already has and adds the only thing missing.
Sintel is distributed by the Blender Foundation with English subtitles inside the file. Here is one of its lines as the film shipped it, and the same line after EveryWord timed it. Both files are served from this site and are in the repository.
What the film shipped
One line, one start, one end. Every subtitle track ever made.
loading ...A renderer given this can show the sentence. It cannot know which word is being spoken, so it cannot highlight one, so it cannot teach anybody to read.
What EveryWord made from it
The same words. A time for each one.
| loading ... |
Now a renderer can light the word being spoken. That is the difference between a subtitle and a reading lesson.
Comparing the two lines ...
This is checked in the browser, by comparing the two files you can also open yourself. It is the safety property, not a nicety: Amazon Transcribe supplies the timings and is never asked what the words are, so a recogniser that hears “their” for “there” cannot put the wrong spelling in front of a child learning to read. The line breaks are the subtitle author's too. On this film, 74 of 75 words anchored to a recognised word; the rest is interpolated inside its own line, so every word is on screen while its line is.
The subtitle file that came with the film · The caption document made from it · Everything the recogniser heard
On your Fire TV
The same reader, ten feet from the screen
Children watch television. EveryWord turns that screen into reading practice with the same captions you can try above, rebuilt for the living room: large type, a remote instead of a mouse, and a focus ring you can always see from the sofa.
The television is a client of the same MCP server Alexa+ uses
record_reading_session describes itself, in the tool list the deployed server hands to any assistant, as the thing the player calls as the karaoke cursor advances. For a while no player called it, so the only progress an assistant could read was progress an assistant had written. Fire TV and Alexa+ were two products sharing a package.
Now the TV app opens a session against the same public endpoint, over the same Streamable HTTP transport, with no shared secret and no private back door, and reports the words the cursor actually passed. Ask Alexa+ how far a child got and the answer comes off the television. If the network is gone the reader keeps working and the row is simply lost; a child in a living room never sees an error.
What the remote does
Not just the on-screen buttons. A Fire TV remote sends real media keys, and the first version of this app ignored every one of them, which reads as broken from a sofa. Play/Pause starts and stops, rewind and fast-forward move five seconds, and Back returns to the library rather than leaving the app mid-story. Everything is reachable with the D-pad alone and focus is always visible.
Put it on your own Fire TV
1Let the Fire TV accept an app you install yourself
On the TV: Settings, My Fire TV, Developer options. Turn on ADB debugging and Apps from unknown sources. If Developer options is missing, open About and select the device name seven times.
2Find the TV on your network
Settings, My Fire TV, About, Network. Note the IP address.
3Install EveryWord from a computer on the same network
Download the APK below, then:
adb connect 192.168.1.20 adb install everyword-tv-v0.3.2.apk4Open it from the Fire TV home screen
EveryWord appears under Your Apps. The remote is the whole interface: every control is reachable with the D-pad, and focus is always visible.
Where this actually stands
- DoneBuilt for Fire OS as a multi-architecture APK, published
- DoneWhole library on the TV, D-pad and remote media keys, progress reported to the MCP server
- DoneRun on an Android TV virtual device, Amazon's documented emulator path
- DoneInstalled, run and filmed on an Amazon-hosted Fire TV (Appstore Quality Central, FOS 14 3P TV)
- Not yetRun on physical Fire TV hardware
Fire OS differs from stock Android TV, most of all in WebView. The TV app renders captions in native views rather than a WebView, which is where the known differences are largest, but it has not been submitted to the Amazon Appstore and will not be until it has run on real hardware. The full target and test record
Ask Alexa+ for a story
An assistant can choose a story that fits a bedtime, skip what a child has already finished, explain a word they tripped on, and record how far they read. It does all of that through six tools on EveryWord's MCP server, which you can talk to from here.
A live session with the reading agent
Alexa+ talks to tools through the Model Context Protocol. Press this and your browser will hold a complete session with EveryWord's server, the way an assistant would when a parent asks for a story. Every row below is the server's own answer.
Your own key
The server is open, so you can try everything above with no key. If you want a space of your own, take a key: free, no account, and readers you name with it are yours alone. Nothing about the key is stored, so keep it.
What EveryWord knows about your child
Five fields, and no microphone. This is the section a parent reads first, so it is the short one.
This is a product for children, so the honest answer about their data should be short enough to read in full. It is five fields.
Everything a session records
readerIdwhichever name the household chose, which need not be the child'sslugwhich story, by its short namewordsFollowedhow many words the cursor actually reachedcompletedwhether they got to the endatwhen
That is the whole row. You can read the code that writes it in apps/mcp/src/mcp.ts.
What it never has
- Audio of any kind. There is no microphone in the web app, the TV app or the server.
- Video, or a camera of any kind.
- A child's name, age, school, or anything that identifies them, unless a parent types one into readerId.
- What they said, because nothing here ever listens.
- A reading level, a score, or a judgement about the child.
The reason is structural rather than a promise. EveryWord reads to a child and watches where its own cursor got to. It never needs to hear them, so it was built without the ability to. A product that cannot listen cannot leak what it heard.
The questions a parent asks
Does it listen to my child?
No, and it cannot. There is no microphone code anywhere in the web app, the TV app or the server. EveryWord reads aloud and follows its own cursor; it never needs to hear a child, so it was built without the ability to.
Will this teach my child to read?
On its own, no, and anyone who tells you otherwise is selling something. Reading along with synchronised text and speech is a well-studied support for word recognition and fluency, and that is what EveryWord is: practice that happens during screen time a child was going to have anyway. It is not a curriculum, not a tutor, and not a replacement for reading with an adult.
How accurate is the highlighting?
Measured rather than asserted. Against gold forced alignments, the end-to-end highlight lands at 30 ms median onset error, with no word lighting up more than 150 ms early across 766 matched words. The method and the raw numbers are in docs/EVAL.md. A word that lights up early is worse than one that lights up late, because it tells a child the wrong word is being spoken, which is why that number is reported separately.
Has it run on a real Fire TV?
Not yet, and the status list above says so rather than implying otherwise. The app is built for Fire OS as a multi-architecture APK and has run on Amazon's documented Android TV virtual device. Fire OS differs from stock Android TV, most of all in WebView behaviour, and EveryWord renders captions in native views rather than a WebView, which is the area of largest known divergence. It would not go to the Amazon Appstore until it had run on hardware.
What does it cost to run?
The expensive part happens once. Speech and word timings are generated per story, not per child and not per play, so a story costs what it costs the first time and nothing after that. The reading itself is a static file and a cursor.
Can I use it without Alexa or a Fire TV?
Yes. The reader above is the whole product and it runs in this browser. Fire TV is where it belongs in a living room, and the MCP server is how an assistant reaches it, but neither is required to read a story.
Where do the stories come from?
Aesop, who has been in the public domain for roughly two and a half thousand years. Nothing here uses copyrighted text, and the library is a file in the repository you can read.
For developers
Put word-by-word captions in your own app
The renderer is its own package, MIT licensed and on npm. A caption document and a playback time go in; the current line comes out with the spoken word highlighted. The web reader above and the Fire TV app import the same package.
npm install karaoke-captions-reactIn a browser
import { KaraokeCaptions } from "karaoke-captions-react";
<KaraokeCaptions doc={captionDoc} time={video.currentTime} />On a TV, with React Native
import { KaraokeCaptionsNative } from "karaoke-captions-react/native";
<KaraokeCaptionsNative doc={doc} time={position} highlightColor="#ffd34d" />karaoke-captions-react on npm · @everyword/captions-core · The caption format