Keyframes and Bitrate

A live encoder produces a wildly uneven stream of bytes, and the rest of the pipeline spends its time pretending otherwise. Ask for three megabits per second and you get three megabits per second on average. Look at it frame by frame and the average is a thing that happens to nobody.

A keyframe is a whole picture, coded on its own, decodable by someone who just tuned in. Everything between keyframes is only the difference from its neighbours, and on a static scene a difference is almost nothing. Point a camera at an empty corridor and the frames between keyframes cost a few kilobytes each while the keyframe costs a hundred.

That ratio is the whole design space. Long keyframe interval, low average bitrate, but a viewer who joins waits for the next one before seeing anything. Short interval, quick joins and quick seeks, and you pay for a full picture several times a second.

Segmented delivery makes the choice for you. A segment has to begin with a keyframe, or the player cannot start decoding at its first byte. So the keyframe interval sets the smallest segment you can cut, the segment length sets how many segments the player buffers, and the buffer sets the latency. Two-second segments come from two-second keyframe intervals, and low-latency modes exist because someone wanted to break that chain.

The buffer is what absorbs the unevenness. It only has to survive the worst spike, not the average one — and it drains at whatever the network gives you, not at whatever the encoder asked for.

The failure everyone remembers is a stall, and a stall is almost never the moment the bandwidth dropped. It is the moment the buffer reached zero, which is somewhere between four and thirty seconds later, on a graph nobody was watching.