search

mbebenita / Broadway

A JavaScript H.264 decoder.
Other
2.73k stars 424 forks source link

Live Video? #181

Open winkmichael opened 6 years ago

winkmichael commented 6 years ago

Any suggestions on the best way to approach implementing this for live video?

soliton4 commented 6 years ago

just do it

germain-gg commented 5 years ago

I'm a bit unsure about how the decoder really works but if you look at formats like HLS or DASH you have different chunks and they can be H264 in a TS container (for HLS case).

If you look at libraries like hls.js they demux chunk before inserting the raw frame data in a SourceBuffer (part of Media Source Extensions).

I wonder whether this would be applicable to Broadway, but if it is that would mean that it could be used for very very long videos but since the data is chunked the playback would start right after the first fragment is loaded.

If anyone has a bit more insight regarding the Decoder architecture I'd love to hear about it

matijagaspar commented 4 years ago

Check my repo (https://github.com/matijagaspar/ws-avc-player), I use Brodway player to play lowlatency h264 stream transfered via websocket.

It's video only (sound is WIP). Basically the beauty of ws + broadway is that you don't need any additional packet wrapping/containers. WebSocket provides a reliable ordered transfer of packets, raw h264 is already divided by NALUnits. So all you need is split the h264 stream by NAL separator and send each NAL unit over WS, than feed each received packet to Brodway.

Of course there are downsides doing this in the "fight against the web" manner, but that's a topic for another day ;)

soliton4 commented 4 years ago

you need to encode with low latency ofc, just as @matijagaspar and @gsouquet wrote as long as you feed complete(!) nal units 1 by 1(!) you will get a decoded output as soon as its available. i experienced some oddness with openh264 encoding where the first 4 frames could not be decoded immediately but after that it was a proper stream. be prepared for encoder hickups. not every nal unit is guaranteed to deliver a frame and not every frame is guaranteed to come from one single nal unit. as far as i know you can have multiple frames with one nal as well.