search

vortexntnu / vortex-auv

Software for guidance, navigation and control for the Vortex AUVs. Purpose built for competing in AUV/ROV competitions.
https://www.vortexntnu.no/
MIT License
85 stars 20 forks source link

ORB-SLAM3 with own camera feed #262

Closed chrstrom closed 2 years ago

chrstrom commented 2 years ago

Estimated time: 10hr Deadline: 10. october

Description: ORB-SLAM3 has been ran using an example video file, but we want to run it with our own camera feed from the ZED camera 😄

jae-hyeong-hwang commented 2 years ago

https://github.com/raulmur/ORB_SLAM2/issues/205 stuck in this issue. Tried to specify directories of libraries, followed the suggested solution but failed.

KristianSem commented 2 years ago

On the Xavier we got ORB_SLAM3_Fixed repo working https://github.com/shanpenghui/ORB_SLAM3_Fixed#%E4%B8%89run-orb-slam3-in-shell It was built with:

ORB-SLAM3 is working in Stereo mode without the use of IMU, which kinda confirms that the problem is in the IMU data stream. The generated map is a mess because of lack of odometry.

It's important to set the online rectification arg to false and use the ZED rectified image topic (instead of the raw image topic), as you won't be able to generate a map otherwise.

ORB-SLAM3-Babbbyyyy

chrstrom commented 2 years ago

Great work 😄 Since this has been achieved, and the issues now have to do with the IMU more than anything, we can close this issue. Documentation is in the OneNote, but will be moved to the internal wiki!

crose72 commented 5 months ago

Hi @chrstrom, I am using an NVDIA Jetson Nano and a CSI ribbon camera (regular monocular and stereo inertial) and I'm having some trouble getting ORB SLAM3 to work with my live camera feed. The issue I have is that I can't enable the visualizer to see how SLAM is working on that video footage. I got the EuRoC example with the visualizer enabled working. My code will run, but it seems that the video feed isn't being passed to the SLAM system correctly and therefore the map isn't being created. Do you think you could help me getting this code up and running? The internet really only shows off what SLAM is and doesn't have many resources to help you get started with your own camera lol.

martongonczy commented 2 months ago

Hi @crose72,

I have the same issue. Have you been able to get it to work?

crose72 commented 2 months ago

Hi @crose72,

I have the same issue. Have you been able to get it to work?

Yes, I am at least now actually seeing the video frames in the viewer and the mapping/localization is working. You wanna know what I changed in my code? Nothing lol. I just switched to a different jetson nano and then it worked all of a sudden. Try this code https://github.com/crose72/OperationSquirrel/tree/master/scratch/SLAM-examples. Specifically the live-slam-example and live-slam-example-2. Takes a second to load up, but it's now working out of the box for me. I didn't calibrate the camera yet though, I just wanted to see the slam working so I'm using a borrowed yaml file.

martongonczy commented 2 months ago

Hi @crose72, I have the same issue. Have you been able to get it to work?

Yes, I am at least now actually seeing the video frames in the viewer and the mapping/localization is working. You wanna know what I changed in my code? Nothing lol. I just switched to a different jetson nano and then it worked all of a sudden. Try this code https://github.com/crose72/OperationSquirrel/tree/master/scratch/SLAM-examples. Specifically the live-slam-example and live-slam-example-2. Takes a second to load up, but it's now working out of the box for me. I didn't calibrate the camera yet though, I just wanted to see the slam working so I'm using a borrowed yaml file.

Thank you for your help. I will check it out. It started working for me as well, it seems like that the visualiser needs some time to start also I need to start moving the camera around otherwise the mapping does not start on the visualiser.