search

realizator / stereopi-tutorial

StereoPi OpenCV depth map examples
GNU General Public License v3.0
112 stars 37 forks source link

green static when recording #7

Open deepav-ai opened 5 years ago

deepav-ai commented 5 years ago

Raspbian Buster image from the wiki page. Raspberry Pi v2.1 cameras. I ran apt-get dist-upgrade so I have to run sudo vcdbg set awb_mode 0 when I startup or it freezes.

Every time I record I get the green stuff to the right of each image. You'll see it in the h264 file.

https://drive.google.com/open?id=1qTC_QbfcZvDPx3c51hKO-YB4DVVkVP7e

`import picamera

camera = picamera.PiCamera(stereo_mode='side-by-side') camera.start_recording('video1.h264', format='h264') camera.wait_recording(5) camera.stop_recording() `

realizator commented 5 years ago

@deepav-ai I see you are not put camera settings here (like resolution and bitrate). THis way system will try to use default settings, that is not good. Try to use direct resolution and FPS settings, like in our code.

deepav-ai commented 5 years ago

It's the resolution ratio that causes that. What is the highest resolution that can be recorded?

I tried camera.resolution = (3840, 1080) which should be two 1920x1080 side-by-side but it said invalid resolution. 1920,540 worked.

deepav-ai commented 5 years ago

I just tried 2560x720 which would be the minimum resolution to be recording HD (side-by-side 1280x720) and there's no error but the script hangs.

I also wanted to try recording each camera separately and that hung so I tried just a single camera with this code and this also hangs:

camera1 = picamera.PiCamera(camera_num=0) camera1.resolution = (1280, 720) camera1.framerate = 30 camera1.vflip = True camera1.start_recording('video1.h264', format='h264') camera1.wait_recording(5) camera1.stop_recording()

realizator commented 5 years ago

@deepav-ai Attempts to use OpenCV and Python for just capturing and recording video is a very bad idea. The better way to do it is to use raspivid utility. Also please notice, that Raspberry Pi H264 encoder unable to encode frame lager than 1920x1080 (hardware limitations). You can read some tech specs here.

PiCamera is relatively slow, and able to capture ~20 FPS at 1280x480, and it depends on your settings. Use this for computer vision, not for high-resolution video recording.

If you want to take a maximum from combination of Python and camera, you need to use some advanced tricks, like mixing Python with raspivid (or raspividyuv) and piping data. Here is an example of such approach: https://gist.github.com/CarlosGS/b8462a8a1cb69f55d8356cbb0f3a4d63 We plan to use this approach in our next Python examples for advanced users.

deepav-ai commented 5 years ago

I don't really need OpenCV as I'm going to do depth maps offline. Just trying to record the highest resolution I can. Two separate video files would be fine as long as they are synced. I was able to get the above code to run after a reboot (without the sudo vcdbg set awb_mode 0) and it recorded.

So I had it run start_recording in two different threads and they are off by about 0.08 seconds which is about 3 frames at 30 fps. Not horrible but not brilliant. GPU seemed to have no problem keeping up. Any idea how I might go about syncing those better?

realizator commented 5 years ago

The feature of stereoscopic mode is cameras syncing. Two different threads in your case will not give you this sync. Also, if you'll check real FPS of recorded files, it can be lower than expected.

In any case, vide recording using raspivid will be definitely much more effective than any Python-based solution.

deepav-ai commented 5 years ago

So 1280 x 480 (640x480 per camera) is the max resolution that can be recorded with a python script? Seems weird that the GPU can keep up with two separate 1080P streams non-synched but can only handle 1/4 of that when synced?

I really need this running in python for integration with other sensors and libraries.

realizator commented 5 years ago

Ok. Please look at the gist code I've provided. You can find a solution of how to control running of very optimized raspivid from your Python code. I think this is the best approach for your case, when you need both high-speed video record or stream (with no processing) and Python control for your robot.