[OTHER] Problems video reading using OpenCV and picam2

[marghelo94]

I am using a HQ camera and a raspberry pi 4. My code acquires 10s-video using picamera2.start_recording and picamera.stop_recording, with the suggested video_configuration and H264 encoder. Then, I convert the video from .h264 to .mp4 format. So far so good. The problem starts when I call OpenCV videoCapture to read this mp4 video. The cap.read( ) function freezes at the first frame. I think there are any interference with the picam2 library, because the same code worked with picam. The acquired mp4 video is not corrupted, and if I use cap.read() on the same video but in other script it works. So, the problem seems to be in the subsequent use of picam2 and opencv. Any suggestion?

Thank you very much

[davidplowman]

Could you include a minimal code snippet that shows how you are expecting to read frames from the file, and which is not working? Thanks!

[marghelo94]

This is part of the code. I call the function to record video in a thread, then I close the camera and I use opencv to read video.

`def camera_recording(picam2,pin):
    GPIO.output(pin, GPIO.HIGH)
    picam2.start_preview(preview=True)
    time.sleep(2)
    picam2.start_recording(encoder,videopath + video_h264)
    time.sleep(10)
    picam2.stop_recording()
    picam2.stop_preview()
    GPIO.output(pin, GPIO.LOW)

picam2 = Picamera2()
controls = {"ExposureTime": 450, "AnalogueGain": 2.0, "Contrast": 2.0}
video_config = picam2.create_video_configuration(controls=controls)
picam2.configure(video_config)
encoder = H264Encoder(10000000)

try:
    while True:
        video_h264 = sample + '_' + datetime.now().strftime('%Y%m%d_%H_%M_%S') + '.h264'
        video_mp4 = video_h264[0:len(video_h264)-4] + 'mp4'

        OUTPUT_VOLTAGE = 700 #starting point
        mcp4725.outputVoltage(OUTPUT_VOLTAGE) 
        time.sleep(5)

        t1 = threading.Thread(target=camera_recording, args=(picam2,Relay_Imaging), name= 't1')
        t2 = threading.Thread(target=slowdown_pump, args=(), name= 't2')
        # starting thread 2
        t2.start()
        time.sleep(5)
        # starting thread 1
        t1.start()
        t2.join() # wait until thread 2 is completely executed
        picam2.close()
        convert(videopath+video_h264, videopath+video_mp4)

        #read video with OpenCV
        cap = cv2.VideoCapture(videopath+video_mp4)
        while (cap.isOpened()):
            ret,frame = cap.read()
            if ret == True:
                print(frame)
                cv2.imshow('Frame', frame)
                keyboard = cv2.waitKey(40)
                if keyboard == 'q' or keyboard == 27:
                    break
            else:
                break
        cap.release()
        cv2.destroyAllWindows()`

[davidplowman]

Hi, thank you for posting the code. Unfortunately I don't seem to be able to run that example so there's not too much that I can suggest.

I think I would take the video_h264 file and see if that looks correct. For example, can you play it with ffplay? If that seems to play OK then the problem may be with this convert function? Obviously I'm guessing here, sorry that I can't be more helpful.

[marghelo94]

``Hi, thank you for your reply. I can play video_h264 and its conversion video_mp4 with VLC media player and they are both ok. I don't understand why the problem in opening videos with OpenCV happens only if I use picamera2 in the same script. If I record a video X in a script and then I read this video X in another script, there is no problem.

I don't think the convert function is responsible of this, I will share below.

`def convert(file_h264, file_mp4):
    command = "MP4Box -add " + file_h264 + " " + file_mp4
    call([command], shell= True)
    print('Video converted!')`

Thank you for the help

[davidplowman]

Hard to know what's wrong. It feels like your h264 stream is probably OK and your mp4 file may be fine too, though mp4 is a fairly complex file format so there's certainly scope for compatibility issues. Maybe you could try a different converter, such as ffmpeg? Something like ffmpeg -i input.h264 -c:v copy output.mp4 might be close to what you want.

[marghelo94]

Thank you for your suggestion. Maybe I've just found the source of error. If I comment picam2.start_preview(preview=True) before picam2.start_recording(encoder,videopath + video_h264) and picam2.stop_preview() immediately after, I can read the recorded video with OpenCV without freezing. It is not clear to me why the preview function makes this effect... Perhaps it isn't the most convenient way to have a preview of video recording... Anyway, the main problem seems to be solved.

Thanks a lot for your help!

[davidplowman]

I'll close this issue for the time being then. Please open up another if you have further questions. Thanks!