search

bluenviron / mediamtx

Ready-to-use SRT / WebRTC / RTSP / RTMP / LL-HLS media server and media proxy that allows to read, publish, proxy, record and playback video and audio streams.
MIT License
11.56k stars 1.46k forks source link

Cache H264 GOPs in order to allow readers to decode frames immediately #1209

Open aler9 opened 1 year ago

aler9 commented 1 year ago

Describe the feature

Discussed here #1132

NOTE: this feature is already present in HLS. It is missing in WebRTC, SRT, RTSP, RTMP.

sourcez commented 1 year ago

This would be a very desired feature for the video platform we are currently building. Any outlook on when this could be build in? Thanks !

jordybsecurite commented 1 year ago

Can you maybe give some pointers as to what you think is necessary to implement this (e.g. which files to edit)? Maybe I can give it a go

aler9 commented 1 year ago

Hello, in my opinion to implement this you can hack formatprocessor.H264, which is an object that is initialized once for every H264 track of every stream:

https://github.com/bluenviron/mediamtx/blob/main/internal/formatprocessor/h264.go

In this structure you have to save all NALUs in a buffer until a key frame is received; when a key frame is received, this buffer must be emptied, and filled with new NALUs starting from the key frame.

When a new RTSP, RTMP of WebRTC client connects (not HLS, since key frame caching is already present in HLS), you have to send to this client all NALUs that are present in the buffer.

0x5e commented 1 year ago

want this feature to! I'm using video doorbell with this project, when doorbell event triggered, video showed up in about 3-4s. a gop cache will make the stream open in realtime.

thanks everyone in advance!!

ethaniel commented 8 months ago

I have a live stream, where I have multiple webcams in one single location (a zoo). I switch between them every 20 seconds. This feature would help make the switch quick and smooth.