Raspivid creates lagging footage when combined with ffmpeg

[kloklojul]

I'm using the following code to pipe the videofootage of raspivid to ffmpeg :

processStringUsb = "/usr/bin/raspivid -o - " + fpsParam + bitrateParam + widthParam + heightParam " --timeout 0 | sudo ffmpeg -r " + str(fps) + " -i - -y -vcodec copy /media/video/" + str(videoFileName) You can assume that all strings and variables are set correctly. I am recording a video onto a usb stick. When i try to pipe a 1080p video with 7Mbit at 25 FPS into ffmpeg the resulting mp4 will have lags (a frozen screen) every ~ 15-20secs which results in either a timeskip or a frozen screen depending if i set the -r Parameter in ffmpeg. In a 2:30min recording i lose roughly 10 seconds of footage. If i record the raw footage to a .h264 format without ffmpeg and mux it afterwards with ffmpeg/mp4box the resulting video has no lags or timeskips. The CPU is never abouve 20%. I can lower the resolution, bitrate (1-2Mbit) and fps to create a fluent video but i can't use these settings since i need "high res" footage to analyze the flow of ground-water so this is not an option. The users on stackechange said this is a bug in raspivid and suggested to open an issue, so here i am.

[JamesH65]

Did the stack exchange commentators say what the bug might be? It looks more like an SD card bandwidth issue to me, or a bandwidth issue somewhere in the processing.

[6by9]

Your use of pipes is most likely the issue.

Linux buffers writes (and reads), therefore writing from raspivid into the pipe does not necessarily trigger ffmpeg to read immediately. You can add the -fl option to the raspivid call to make it request a flush after every write, but even so that is not guaranteed to make ffmpeg read immediately as it depends on how ffmpeg is waiting.

[kloklojul]

Did the stack exchange commentators say what the bug might be? It looks more like an SD card bandwidth issue to me, or a bandwidth issue somewhere in the processing.

Unfortunatly not, they asked for the command line i used to record the mp4 and after i posted it all i got was something along the lines, "looks good ffmpeg reads the input as fast as possible and if raspivids doesn't deliever a stable framerate this is what happens".

In another thread i got the following: "Lower the resolution and check. I don't think your Pi can process the video fast enough. The freeze+speed-up is happening since the playback timing is set using the interval at which ffmpeg receives the frames" But since the CPU is peaking at 20-25% i don't think it should be a performance problem, or am i wrong ? Also where is raspivid piping the output to ? The ram or the storage ? If the output goes to the ram than to ffmpeg and than to the usb stick shouldn't this be just as fast as just writing the raw file to the usb stick ? Don't know much about linux let alone raspivid so sorry if that all sounds dumb.

Your use of pipes is most likely the issue.

Linux buffers writes (and reads), therefore writing from raspivid into the pipe does not necessarily trigger ffmpeg to read immediately. You can add the -fl option to the raspivid call to make it request a flush after every write, but even so that is not guaranteed to make ffmpeg read immediately as it depends on how ffmpeg is waiting.

So if i try it with -fl and it still does not work as expected i'm out of luck ? Or is there another option to create a "on-the-fly" mp4 ? The resulting mp4 will be processed by another program which can't handle raw h264. Having an "on-the-fly" option would greatly increase the efficiency of our current setup.

[6by9]

So if i try it with -fl and it still does not work as expected i'm out of luck ?

If it still does not work, then we look at other potential solutions. Please try what is suggested before asking for what happens after that.

[kloklojul]

I recorded ~ 30 minutes of footage playing around some more with the parameters including adding -fl. The last command i used was sudo /usr/bin/raspivid -o - --width 1920 --height 1080 --framerate 30 -b 9000000 -fl --timeout 0 | sudo ffmpeg -r 30 -use_wallclock_as_timestamps true -i - -y -vcodec copy -r 30 /media/video/testingNOW.mp4 . Quality was nice and adding 'fl' did infact improve the situation but there were still frozen screens and catch ups every ~20 seconds.

[kloklojul]

Also if i remove both -r 30 and -use_wallclock_as_timestamps the video has no stutters but a 20min recording will result in a ~23min long video. After 2-3min the log of ffmpeg will also say that it is only receiving 29 FPS. Same thing happens if i set --framerate to 25, after a while the log says ffmpeg is only getting 24 FPS.

[6by9]

Your use of pipes is still the most likely the issue.

raspivid produces a raw H264 elementary stream (ES). An ES has no frame rate or timestamp information in the stream itself, therefore you are having to tell ffmpeg what frame rate to assume for the incoming data. If that varies (and it will as the sensor and SoC are on different oscillators) then realtime and wall time will diverge, and if viewing in realtime then you'll get a glitch. There is no way to correct this with a pipe.

If you use the V4L2 driver then you get both the stream and timestamps for every buffer. That should make ffmpeg happier. Load it with sudo modprobe bcm2835-v4l2, and then something like ffmpeg -f v4l2 -input_format h264 -video_size 1280x720 -i /dev/video0 -c:v copy foo.mkv will capture via V4L2 with hardware accelerated H264 encode.

Integrating writing to a container (MP4, MKV, AVI, etc) into raspivid is a pain because the API to libavformat is not stable, therefore what works against one release won't work against another.