FPS declaration required even with SPS headers?

[dans98]

I'm passing SPS headers, but i've noticed if the framerate is higher than 30fps latency will continue to get worse as a stream plays. If I declare fps when instantiating jmuxer then the stream plays perfectly fine.

Is fps declaration required even if you are passing SPS headers?

[samirkumardas]

SPS does not contain timing information. You must provide either fps or chunk duration.

[dans98]

SPS does not contain timing information. You must provide either fps or chunk duration.

I'm feeding jmuxer h264 from a raspberry pi camera module via the picamera python framework. and the documentation says the following.

sps_timing - When True the encoder includes the camera’s framerate in the SPS header. https://picamera.readthedocs.io/en/release-1.13/api_camera.html

I'm not familiar enough with the h264 spec to speak authoritatively, but from what I have researched thus far passing the framerate sounds like it within spec, but not common and not required.

[samirkumardas]

Sorry, I meant to say SPS does not contain mandatory timing information. You must provide either fps or chunk duration. Anyway, right now jMuxer expects an fps or chunk duration. If you don't provide either, it will run by default at a constant fps.

[samirkumardas]

However, I am adding it to my TODO list. It could try grabbing timing information from SPS if nothing is explicitly declared during jMuxer initiating.

[dans98]

However, I am adding it to my TODO list. It could try grabbing timing information from SPS if nothing is explicitly declared during jMuxer initiating.

It will be interesting to see how this works, because this might be a good way to deal with variable framerate streams.

[samirkumardas]

You can always provide a chunk duration in every feed, this is how it works with a variable frame rate at the moment. Anyways, I am not sure about how will it go with SPS timing info as SPS is not mandatory for each chunk or doesn't need to provide on a regular basis. I will check when I get free time.

[lrowe]

I believe SPS timing info only give you fps, not the individual frame timings needed for variable frame rate. (From what I can tell these are only possible to capture to a separate file with raspivid.)

-stm, --spstimings: if you put the created h264 file into a container with programs like MP4Box, ffmpeg or gstreamer, the program can read the frame rate from the h264 file and it can set the right frame rate for the output automatically. This is kind of useful all the time and does not disturb the rest of the file.

https://www.raspberrypi.org/forums/viewtopic.php?t=231505

[neodescis]

I'm looking at this myself, as I would really like to not have to provide the fps or duration. In the stream I'm looking at, coming from VLC, the SPS data does have the correct info needed to calculate fps on the fly. I did notice, however, that your SPS parser isn't correctly getting the two fields needed to do this (time_scale and num_units_in_tick), so the issue there will need to be figured out first.

For reference, another library I'm using (to demux an MPEG-TS stream before handing the video stream over to jmuxer) does decode the SPS info correctly:

https://github.com/goldvideo/demuxer/