stereo mode

[wezzoid]

Hi, how do I use this with the Compute Module to publish nodes for 2 cameras?

[realizator]

+1, the same question. Raspbian supports stereoscopic mode out of the box (https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=85012&sid=ec557617fabfa9942e5baeec16bf862c)

Raspberry Pi Compute module supports 2 cameras, and using a Pi devboard or ready-to-use boards like StereoPi you have native stereoscopic image. I can help to develper by providing hardware for tests (StereoPi engineering version). @rohbotics, please contact me if you need hardware set for tests and stereoscopic implementation to your driver.

[rohbotics]

We use the same firmware and kernel as raspbian, so that side of stereoscopic processing should work.

Can you use our image (https://downloads.ubiquityrobotics.com/pi.html) , and see if you can get PiCamera to output stereo?

[realizator]

@rohbotics, it just works! :-) The only thing I did is put dt-blob.bin (device tree for StereoPi) in a boot partition to enable two cameras support. Thank you!

[realizator]

@rohbotics, concerning @wezzoid question - how do you think, what is the simplest way to publish nodes for stereo processing?

[rohbotics]

I'll have to look into the mmal format for stereo, but the way to do it is basically publish 2 separate images and use stereo_image_proc for combining them into a stereo_msgs/DisparityImage in ROS.

A couple concerns:

• I think we will be limited to raw publishing, or will have to find a way to cut a jpeg in half to publish it separately
• Calibration, how do we save calibration data for each camera separately?

Also PRs are welcome :)

[realizator]

As an idea - you can use two cameras separately. In raspivid I use -cs 0 and -cs 1 to use cameras independently and simultaneously (in my case it is front view and rear view on a drone, not stereoscopic mode). The only disadvantage is not so good image synchronization in comparison with -3d single image capture mode.

But as first step for tests it may be Ok. Is it possible to load and use two instances of your driver and two cameras simultaneously? Is this solve problem with separate calibration data for each camera?

[rohbotics]

Being able to do select a camera for each instance of the node would be a really good first step in supporting stereo.

I'll look into it.

[davecrawley]

Stereo is currently supported on the compute module. Can it be made to work on the RPi 3B+? CSI is able to support up to 4 cameras on the same bus so I'd expect to enable stereo cam we'd just need a different design of camera cable - is that correct or am I way off?

David

On Wed, Oct 17, 2018 at 10:08 AM Rohan Agrawal notifications@github.com wrote:

Being able to do select a camera for each instance of the node would be a really good first step in supporting stereo.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/raspicam_node/issues/41#issuecomment-430710766, or mute the thread https://github.com/notifications/unsubscribe-auth/ALDeVjOSubOE0se7CH2BohwWoi41zTkEks5ul2QogaJpZM4S8tkT .

[wezzoid]

I used the picamera Python lib to get stereo working for my project (https://hackaday.io/project/159212-augmented-mekamon-robot) I put the camera in stereo mode, loaded the data in as an array, chopped it in half and published the left & right frames to ROS, basically. Doing it in Python is slow. To get around this I put the camera in top-bottom stereo mode, which means chopping the image data in half is just one 'cut' instead of one cut per row.

Here's my (amateurish) code: https://pastebin.com/RQ4DMBJj

@davecrawley the normal Pi 3 doesn't have the right pins exposed. Only the compute module allows this. Why Raspberry don't want to add a second CSI connector to their revised boards I don't know.

[realizator]

@wezzoid thank you for your sample code! It is right this example that we were looking for! I will try to combine it with @rohbotics camera selection support from his last pull request after tests.

[realizator]

@wezzoid thank you for your code sharing. After minor changes it works fine:

BTW I had tried to contact you on hackaday.io but did not received any answer. Are you still there?

[rohbotics]

@realizator That is very cool! Looks like I am going to have to pick up a StereoPi :smile:

[realizator]

@rohbotics StereoPi will be available after our coming-soon crowdfunding (here is pre-launch page), but I can send you one of my current revision board for experiments right now. I just need your address for parcel. I wrote you email to send2rohan@... but did not receive a response (possibly my message was marked as a spam). You can write me to realizator[at]gmail[dot]com I wonder if it is possible to make stereoscopic driver for ROS like other stereoscopic cameras have. It will be compatible with original Raspberry Pi CM3 devboard and with StereoPi. Also we plan to do StereoPi schematic open source, and I hope some companies like Adafruit will help us to make this board mass-producted and available for DIY enthusiasts.

[wezzoid]

@realizator - awesome stuff! And that StereoPi board looks VERY interesting! I clearly haven't been checking my Hackaday, oops.

[realizator]

@wezzoid, @rohbotics - I put all my files used for tests here: https://github.com/realizator/StereoPi-ROS-depth-map-test May be they will be useful.

[nirvenesh20]

I'm very sorry guys, i'm actually developing a visual for monocular visual impaired patients on which i am developing a depth map, i'd like to experiment with your code but i am not sure how to enable the stereoscopic camera setup on the compute module. would really appreciate that

[realizator]

@nirvenesh20 If you are talking about Raspberry Pi Development board with Compute Module and two cameras - here is the manual from RPi Foundation: https://www.raspberrypi.org/documentation/hardware/computemodule/cmio-camera.md

I'd like to notice that by default the Devboard have 1 adapter for camera and another for connecting display. So you need to buy second adapter to connect two cameras (https://uk.farnell.com/raspberry-pi/rpi-cmdk-adapter/adaptor-compute-module-camera/dp/2433318?st=raspberry%20adapter). Looks like it is out of stock now, so you need to google it.

[nirvenesh20]

@nirvenesh20 If you are talking about Raspberry Pi Development board with Compute Module and two cameras - here is the manual from RPi Foundation: https://www.raspberrypi.org/documentation/hardware/computemodule/cmio-camera.md

I'd like to notice that by default the Devboard have 1 adapter for camera and another for connecting display. So you need to buy second adapter to connect two cameras (https://uk.farnell.com/raspberry-pi/rpi-cmdk-adapter/adaptor-compute-module-camera/dp/2433318?st=raspberry%20adapter). Looks like it is out of stock now, so you need to google it.

Thank You very much for the swift reply I've gotten a PiZero camera cable as an alternative, I'm not sure if it affects in any way , also I'm booting with an image link you provided for ROS and using a Waveshare compute module IO board with CM3L. Any further advice on my configuration?

[realizator]

@nirvenesh20 Ok, now I understand your problem. If you use our image, you also use our dt-blob.bin This file explains to Raspbian how the second camera is connected. So you need to connect wires as explained in Raspberry manual, and put dt-blob.bin from their manual to /boot partition instead of our dt-blob.bin. It should work. The easiest way to test if it is everything Okay with cameras are these 3 steps:

1. Open terminal
2. type raspistill -cs 0 You should see 5 seconds preview from your first camera
3. type raspistill -cs 1 You should see 5 seconds preview from your second camera.

UPD> Just noticed you use Waveshare IO board. I have no any experience with it, but hope it is pin-to-pin compatible with original Raspberry's devboard, and you can use Raspberry's guide.

[nirvenesh20]

@realizator Thank you very much, will try it and keep you informed You're a saviour

[nirvenesh20]

@realizator unfortunately, using your image from https://downloads.ubiquityrobotics.com/pi.html and thedt-blob.bin file from https://www.raspberrypi.org/documentation/hardware/computemodule/cmio-camera.md results in MMAL error

[realizator]

@nirvenesh20 try to do several steps back. There are lot of possible reasons of fail (wires commutation at devboard, bad cameras, bad or wrong ribbon connections etc.)

1. Use clear Raspbian image, without any dt-blob.bin
2. Try to preview picture from one of your cameras. UPD> Do not forget to run raspi-config and turn on camera support before using raspistill on stock Raspbian. UPD2> On stock raspbian without dt-blob.bin use raspistill command instead of raspistill -cs 0. cs option means "Camera Select" and works on two-cameras systems only.
3. If step 2 is Ok, try to connect another camera to the same CSI connector and take second picture.
4. If 2 and 3 is Ok - both cameras and ribbons are ok, go to the next step.
5. Add Raspberry's dt-blob.bin and check both cameras. Notice: please use appropriate dt-blob.bin for your wires commutation on the devboard.
6. If 5 is Ok, copy Raspberry's dt-blob.bin, and try to obtain images from each camera. When it will work, just put the same dt-blob.bin to our image.

[realizator]

@nirvenesh20 By the way I see another wallpaper on your photo. I can suggest you did not use our full linux image, but just our files on Github. That is camera support on your image can be disabled. Try to start sudo raspi-config and check if it is turned on.

[nirvenesh20]

@realizator Thank you for your detailed instructions, everything works absolutely well. I am able to run the publisher as well with no errors. But how do i excute step 3 onwards from https://github.com/realizator/StereoPi-ROS-depth-map-test . If everything works well, I will purchase the Stereopi , when it starts shipping in April . Also i really appreciate your time and effort, its really nice of you to guide a person like me

Also the image i was previously using was the latest image provided on your page published on the 19/2 /2019. However I'm now using an older image from the same page. Thank You

[realizator]

@nirvenesh20 Could you please clarify your question. If you already put all files in your home folder, you need just copy-paste command from step 3 in your console and run it. And, as you may guess, you need to run several console instances (4), and run each command in a separate console window. p.s. Image you use is not my image, but image from Ubiquiti Robotics. :-) I'm one of the StereoPi developers, it is another team. We use this image because it has great support, always updated and works out-of-the-box.

UPD> I added terminal instances notice in my Github repository

[nirvenesh20]

@realizator I did run the command on a different console/separate console window . But i get an error [rospack] Error: package 'stereo_image_proc' not found . I have never got a proper link to search for this package as well.

[realizator]

@nirvenesh20 I think you should install it something like this: sudo apt-get install ros-kinetic-stereo-image-proc

[nirvenesh20]

@realizator thank you very much. Everything worked perfectly. Also is there anyway i could enlarge the disparity map so that it could display in full screen.

UPD >also i think it should be ROS_NAMESPACE=stereo rosrun stereo_image_proc stereo_image_proc instead of ROS_NAMESPACE=stereo rosrun stereo_image_proc stereo_ige_proc

[realizator]

@nirvenesh20

1. Thank you for your typo notice ("ige" instead of "image"), I've fixed it.
2. I'm not a guru in ROS, so you should ask some more advanced guys about full-screen preview :-) I just want to notice, that in a real onboard robot depth map implementation it is better to turn off visualization at all, as it takes a lot of CPU resources.
3. All my experiments was proof-of-concept. For better results I recommend you to do cameras calibration. It will dramatically increase depth map precision and smoothness.

[nirvenesh20]

@realizator I've been very much intrested with the stereopi, but was just curious, if the stereo camera requires any GPIO jumpers to be connected. Its a concern as i plan to attach a 3.5 inch touch display (something like this https://www.cytron.io/p-3.5-inches-tft-touch-screen-for-rpi-3?currency=MYR&gclid=Cj0KCQjwpsLkBRDpARIsAKoYI8w13Jbbnhs0VCPdAhWUcp9P_4unY-9HX9MaTlGX6zm5806LN0B7ey8aAu0iEALw_wcB to the stereopi, and it wouldn't be possible if there's need for GPIO connectors for the cameras

[realizator]

@nirvenesh20 Your question is a bit offtop here (as it is not about raspicam_node). Could you please ask it here on a Raspberry Pi forum? Thank you!

[Lashhev]

I managed to adapt the new mmal api with support for stereo mode. You should try my fork https://github.com/Lashhev/raspicam_node.git (stereo branch)

[rohbotics]

@Lashhev That is awesome! Could you please make a PR so that we can get the necessary changes merged in to this node?

[Lashhev]

@rohbotics, Oh. Ok. In which branch should I make PR?

[rohbotics]

Make the PR to the kinetic branch.

Rohan

On Sat, Nov 16, 2019 at 9:01 AM Lashchev Andrew notifications@github.com wrote:

@rohbotics https://github.com/rohbotics, Oh. Ok. In which branch should I make PR?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/UbiquityRobotics/raspicam_node/issues/41?email_source=notifications&email_token=AAVRY33U3P3TSJF7VBETMDLQUADLDA5CNFSM4EXS3EJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEHTLQQ#issuecomment-554644930, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVRY36V5P6NW7P6OWAZUHTQUADLDANCNFSM4EXS3EJQ .

[tiralonghipol]

hi @rohbotics i followed the discussion and I was wondering if the stereo ros node can be applied to this stereo hat (https://www.arducam.com/product/b0195-synchronized-stereo-camera-hat-raspberry-pi/) can you give a feedback on this ?

Thank you in advance!

[JesseCake]

hi @rohbotics i followed the discussion and I was wondering if the stereo ros node can be applied to this stereo hat (https://www.arducam.com/product/b0195-synchronized-stereo-camera-hat-raspberry-pi/) can you give a feedback on this ?

Thank you in advance!

Yes definitely!