[HOW-TO] Get rid of missing frames while set up circular buffer

[CodeTim24]

Hey, i'm using the Circular Buffer with Pi5, Cam V3 and a fresh Bookworm installation. I realised some missing frames and the wrong video length during set up some different specifications to the cam and the circular buffer.

Background Im trying to use the Circular buffer to safe the past X seconds after an event. While debugging my code i realised the wrong video length after converting the h264-file to mp4 with MP4Box. For understanding the problem i created a loop that set up different durations every time (code at the end).

Problem After setting up the camera (2304 x 1296) with an duration of 5 sec & 30fps for the circular buffer i got a video of 4.9 sec. and 148 frames recorded. The longer the duration the fewer frames are recorded (e.g. 9 sec duration (30fps) -> buffersize: 270 -> 240 frames recorded -> video length 8.1 sec)

When using higher quality (4608 x 2592) the number of lost frames increases: e.g. (9 sec duration (10fps) -> buffersize 90 -> 54 frames recorded -> video length 5.4 sec). The speed of the converted video to mp4 is way to fast. I think that MP4Box works with the recorded frames and 10 fps as i set up but the missing frames cause the wrong video duration and speed.

Another problem is the missing timestamp.txt file for checking the timestamps. How do i set up the pts='timestamp.txt'using the circular buffer?

Thoughts

• i've found that the camera need some time to set up, i added the time as cam_start_up_time=2 before filling up the circular buffer
• i added some extra buffer as buffer_size= int(frame_rate*(dur+0.2) but it doesent affected the stored amount of frames as i wanted
• i thought that while recording and storing, some frames in the circular buffer are missing. That causes an non fillled buffer at the end.

Question How can i record the right duration and the right speed for different camera modes (2304x1296 & 4608x2592) and fill the buffer with the right amount of frames?

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import CircularOutput
import subprocess
import time

i = 0
dur= 5
cam_start_up_time= 2

while i <= 4:
    frame_rate = 40

    print("i: " +str(i))
    print("dur: "+str(dur))

    # Configure camera
    camera = Picamera2()
    config = camera.create_video_configuration(main={"size": (2304,1296)}, controls={"FrameRate": frame_rate, "NoiseReductionMode": 1})
    camera.configure(config)

    # Start camera
    camera.set_controls({"AfMode": 1, "AfTrigger":0})
    camera.start()

    encoder = H264Encoder(bitrate=10000000, iperiod=frame_rate)
    buffer_size= int(frame_rate*dur)
    print("\nBuffersize: " + str(buffer_size) +"\n")
    output = CircularOutput(buffersize=buffer_size)

    #filling Buffer
    camera.start_recording(encoder, output, pts='timestamp.txt') 
    time.sleep(dur+cam_start_up_time)

    #start Recording
    print("\nrecording\n")
    output.fileoutput = f"video/test{i}.h264"
    output.start()

    output.stop()
    camera.stop_recording()
    camera.close()

    #conversion MP4
    output_file = "video/test"+str(i)+".mp4"
    fps = frame_rate
    input_file = "video/test"+str(i)+".h264" + ":fps="+ str(frame_rate)
    command = ["MP4Box", "-add", input_file, output_file] 
    try:
        subprocess.run(command, check=True)
    except subprocess.CalledProcessError as e:
        print("Error MP4Box")

    #duration
    input_file = "video/test"+str(i)+".mp4"
    command = ["ffprobe", "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height,duration,bit_rate", "-of", "default=noprint_wrappers=1", input_file]
    try:
        subprocess.run(command, check=True)
        print("\n")
    except subprocess.CalledProcessError as e:
        print("Error ffprobe")

    #captured frames
    input_file = "video/test"+str(i)+".h264"
    command = ["ffmpeg", "-i", input_file, "-c", "copy", "-f", "null", "/dev/null"]
    try:
        subprocess.run(command, check=True)
        print("\n")
    except subprocess.CalledProcessError as e:
        print("Error ffmpeg")

    dur+=1
    i+=1

[davidplowman]

Hi, I think mostly the problem is that you're asking it to encode 2304x1296 @ 40fps which is beyond its capabilities. It will work better if you change the main stream format to YUV420 as this will save a pixel format conversion (so use main={"size": (2304,1296), "format": "YUV420"}) but even with this it won't achieve what you want.

My rule of thumb is that so long as there's not much else going on, you should get "close" to 1080p50. So you should certainly be able to manage 1080p40, or 2304x1296 @ 30fps. (2304x1206 is 44% larger than 1080p.)

A few other minor tips: when it drops an I-frame from the start of the circular buffer, it has to drop all the P-frames up to the next I-frame. So you generally find up to iperiod fewer buffers there than you might have expected. Also, by stopping the output before stopping the encoder you'll be losing any frames still queued up in the encoder. So maybe just drop the output.stop() - the output will be stopped anyway once the encoder is done.

[CodeTim24]

Thank you su much! It works way better. Could you tell me how to add the timestamps.txt while using circular output, too?

[davidplowman]

Unfortunately I don't think the CircularOutput class has ever been made to support timestamps properly. It looks like it would need to store them in the fifo so that it can write them out later. I don't know if that's something you'd know how to change...?

[CodeTim24]

i'll have a look. Thanks