Frame missing on RPI

[mamtesh-grydsense]

I have done a simple experiment to ensure that there is no frame missing from the capture of the Arducam Tof camera. But there are frames missing for DEPTH data. I have just dumped and plotted the time difference between frames and seen the many picks in the interval plot. Please have a look at to plot and kindly give a workaround to fix this frame-missing issue.

Attaching the code snippet as well to reproduce it.

`import cv2 import time import numpy as np import ArducamDepthCamera as ac

def capture(): cam = ac.ArducamCamera() if cam.init(ac.TOFConnect.CSI,0) != 0 : print("initialization failed") if cam.start(ac.TOFOutput.DEPTH) != 0 : print("Failed to start camera")

# cam.setControl(ac.TOFControl.RANG, 4) 

frm_cnt = 0
durtn = 0
prev_time = time.time()*1000
diff_lst = []
while True :
    t1 = time.time()
    frame = cam.requestFrame(200)
    if frame is None:
        break
    # depth_buf = frame.getDepthData()
    # amplitude_buf = frame.getAmplitudeData()
    cam.releaseFrame(frame)
    curr_time = time.time()*1000
    diff = curr_time - prev_time
    diff_lst.append(diff)
    prev_time = curr_time
    frm_cnt += 1
    durtn += (time.time() - t1)
    if durtn>=120: # time in second
        print("frm cnt: {}".format(frm_cnt))
        break

arr=np.asanyarray(diff_lst)
np.savetxt("diff_time_2_min_raw.csv", arr, delimiter=",")
print("done")

cam.stop()

if name=='main': capture()`

[dennis-ard]

Thank you for sharing your experiment and code snippet. I tried your code and reproduced the results. However, I added code to measure the frame rate and found that the frame rate fluctuates in the range of 1 to 2 frames per second. We think this is normal.Since the depth data is calculated by collecting four consecutive frames of RAW data by the camera, sometimes frames may be dropped due to discontinuous images, resulting in frame rate fluctuations. You can also add the following code to the loop of the capture function to see the frame rate fluctuate every second.

def display_fps():
    display_fps.frame_count += 1
    current = time.time()
    if current - display_fps.start >= 1:
            print("fps: {}".format(display_fps.frame_count))
            display_fps.frame_count = 0
            display_fps.start = current

display_fps.start = time.time()
display_fps.frame_count = 0

def capture():
  ...
  while True :
    ...
    if durtn>=120: # time in second
        print("frm cnt: {}".format(frm_cnt))
        break
    display_fps()
...

Below is the result of my test

pi@raspberrypi:~/Arducam_tof_camera/example/python $ python diff.py
fps: 22
fps: 30
fps: 30
fps: 30
fps: 29
fps: 30
fps: 30
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 30
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 30
fps: 31
fps: 31
fps: 30
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 30
fps: 31
fps: 31
fps: 29
fps: 30
fps: 29
fps: 29
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 29
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 29
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 30
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 30
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 31
fps: 30
frm cnt: 3569
done

[luveti]

Hello, how do you go about detecting discontinuous sequences of raw frames?

My company is using V4L2 to directly read the raw frames, but the sequence number returned by the kernel does not seem to always line up. Sometimes the first frame is a different phase then it should be.

We take the modulo of the sequence number to determine which of the 4 frames it is.

Is there an algorithm we could use to determine if the 4 raw frames are in the proper order?

[Dion4cen]

We use the timestamp obtained by the frame to determine whether it is a continuous frame. The Tof camera will output 4 frames in a row (in a short period of time), and the next 4 frames will be very long (8.3ms if it is 120fps)

[luveti]

That seems to have fixed the issue. Thank you!