ffmpeg stops rendering if there isn't a unbroken sequence of frames

[FrYakaTKoP]

ffmpeg needs a unbroken sequence of frames to render or it will stop.

as example: frame000001.jpg frame000002.jpg <--- this is the last frame which gets rendered because frame000003.jpg is missing frame000004.jpg

this comes from the fact timelapse passes frame%6d.jpg to ffmpeg or "that how ffmpeg works"

ffmpeg does support -pattern_type glob but this will also not work for us here. I need to find another solution to this Problem. Atleast after some quick tests, here the full test command i used

/usr/bin/ffmpeg -r 3 -f image2 -pattern_type glob -i '/tmp/timelapse/frame*.jpg' -threads 2 -g 5 -crf 23 -vcodec libx264 -pix_fmt yuv420p -an  '/tmp/timelapse/timelapse__20230728_2306_test.mp4' -y

I don't have much time to find a solution for this issue, but timelapse itself should never skip a frame number without successful collection of a frame. Only reason i see this can be a problem if a user likes to delete a certain amount of frames in the middle to protect his pivacy or similar, so this needs to be addressed but with lower urgency then others.

Feel free to comment under this if you have a (ideally tested) solution to this issue.

(This issue was orginally raised by @Yerusalmi as PR #117 , but his suggestion/solution doesn't work)

[Yerusalmi]

@FrYakaTKoP I wanted to make minimal change but it wasnt right.. sorry for that! Here is what I have and what works for all my prints:

ffmpeg version 4.3.6-0+deb11u1+rpt1 - built with gcc 10 (Raspbian 10.2.1-6+rpi1) /usr/bin/ffmpeg -f image2 -pattern_type glob -r 25 -i '/tmp/timelapse/*.jpg' '/tmp/timelapse/timelapse.mp4'

I just tried it with all other parameters that you have and it created the timelapse video with all available frames (frame00002.jpg was missing): /usr/bin/ffmpeg -r 10 -f image2 -pattern_type glob -i '/tmp/timelapse/*.jpg' -threads 2 -g 5 -crf 23 -vcodec libx264 -pix_fmt yuv420p -an '/tmp/timelapse/timelapse.mp4' -y I will be using this one instead of the simplified version since I see it works.

Regarding timelapse skipping frame, sometimes I get a red box saying ERROR TAKING FRAME in the fluidd UI. I will debug the message next time I see it.

[FrYakaTKoP]

would be great to have a moonraker,log when a lost frame happens. Again Timelapse shouldn't continue counting/add a new frame if collection of the last didn't succeed. That's maybe a bug which needs to be addressed. Thanks for providing a full command line, i will play around with it later.

[Yerusalmi]

Hello @FrYakaTKoP, I got one error of a lost frame. Below is from moonranker log where it failed:

2023-08-12 13:18:47,475 [shell_command.py:_check_proc_success()] - Command (wget http://localhost:8080/?action=snapshot -O /tmp/timelapse/frame000022.jpg) cancelled
2023-08-12 13:18:47,475 [timelapse.py:newframe()] - getting newframe failed: wget http://localhost:8080/?action=snapshot -O /tmp/timelapse/frame000022.jpg

[jfurner]

Appreciate the discussion and solutions here. I had two layers of problems related to this:

• generated timelapse videos would almost always be cut short, not including all collected frames. Investigation into this revealed that intermittently there would be a captured image file with size 0 bytes. I believe this was due to a camera issue, possibly when a capture attempt was made when the camera stream had dropped out. When ffmpeg reaches the 0 bytes image file, it generates the video file up to that frame but no further, with no obvious error or warning of what had happened. This resulted in random length, but always too short timelapse videos being generated, always up to the first 0 byte image file.
• I fixed this issue by adding to timelapse.py a shell command to remove the 0 byte files before calling ffmpeg:

           # remove 0 byte files
          cmd = "find /tmp/timelapse -size 0 -delete"
          logging.info(f"Remove 0 byte files: {cmd}")
          shell_cmd: SCMDComp = self.server.lookup_component('shell_command')
          scmd = shell_cmd.build_shell_command(cmd)
          try:
              cmdstatus = await scmd.run(verbose=True,
                                          log_complete=False,
                                          timeout=9999999999,
                                          )
          except Exception:
              logging.exception(f"Error running cmd '{cmd}'")

• However this change now created the issue described here - I had deleted some frames, so there was an unbroken sequence.
• Modifying timelapse.py to create the command as described above, with -pattern_type glob -i '/tmp/timelapse/*.jpg' fixed that problem: filelist = sorted(glob.glob(self.temp_dir + "frame*.jpg"))

          # build shell command
          cmd = self.ffmpeg_binary_path \
              + " -r " + str(fps) \
              + " -pattern_type glob" \
              + " -i '" + inputfiles + "'" \
              + filterParam \
              + " -threads 2 -g 5" \
              + " -crf " + str(self.config['constant_rate_factor']) \
              + " -vcodec libx264" \
              + " -pix_fmt " + self.config['pixelformat'] \
              + " -an" \
              + " " + self.config['extraoutputparams'] \
              + " '" + self.temp_dir + outfile + ".mp4' -y"

This is all probably fairly specific to me as the 0 byte files were a result of a camera USB issue in the first place. In the end I moved my camera to a different USB port and the 0 byte files went away.