search

introlab / rtabmap

RTAB-Map library and standalone application
https://introlab.github.io/rtabmap
Other
2.71k stars 775 forks source link

D455 vs. D435i vs L515 for 3D Room scanning #614

Open peta-peta opened 3 years ago

peta-peta commented 3 years ago

Dear all, I want to scan rooms and create water tight meshes. I have started exploring intel realsense L515 with RTABMAP-Ros but I am not happy with the results. Mapping is very unstable and sensitive to daylight. Since I can see many successful applications using realsense D435i I am thinking to buy one, too, and use it directly with RTABMap. Today i learned that Intel has already released a new version D455 and I am wondering if any of you has some experience with it using RTABmap for mapping and odometry. Which one shall I buy, or do you suggest even a different camera or a combination of different devices for room scanning with RTABMap - Room has to be a watertight model with floor, walls, windows and ceiling.

Cheers Peter

Anderson191919 commented 3 years ago

I need the first error in the log to debug. The one starting with "Given stream configuration is not supported by the device!"

No need to explicitly set 640x480x30 anymore in the Preferences dialog, in the code I just search for 640x480x30 when L500 is detected.

What very strange is, is that although rtabmap detects a L500 it is not adjusting the pixel and frequency automatically :/

Anderson191919 commented 3 years ago

Is rtabmap trying to use my laptop cam?

Anderson191919 commented 3 years ago

Sorry that I am frequently writing. Another problem could be maybe windows, because it is detecting the L500 as two seperate inputs in the device manager: grafik

In comparison an azure kinect - which works properly in rtabmap - is displayed as one device: grafik

What is your opinion? Thank you very much in advance

matlabbe commented 3 years ago

Based on the error log, it looks like only the RGB camera is detected at the device ID "0". However, I would have expect that 640x480 would still exist even if only the RGB camera is detected (here the minimum resolution is 960x540). If you set device ID to 1 or 2 in Preferences->Source, would you have a different output?

What is the version or your RealSenseViewer? Does your sensor have the latest firmware version? What is it? (RealSenseViewer->Info).

@rlabs-oss Can you launch rtabmap with Logger level set to Info to show those logs: https://github.com/introlab/rtabmap/blob/80199f23b59c3982b70ebb3b2ad412067a2eca3e/corelib/src/camera/CameraRealSense2.cpp#L682-L689. I would like to compare which video profiles are detected in Linux in comparison to Windows above. Also, what is the firmware installed on your L515? Thank you so far for this debugging!

EDIT: Looking at this post, it seems there is no 640x480 RGB profile... not sure how the current code is working on Linux?! Well, there they say 640x480 works?!

rlabs-oss commented 3 years ago

@rlabs-oss Can you launch rtabmap with Logger level set to Info to show those logs:

See output 640x480x30 is a valid RGB mode. I will download the win version and test it on the same machine running win10. and post the same info shortly...

Anderson191919 commented 3 years ago

Based on the error log, it looks like only the RGB camera is detected at the device ID "0". However, I would have expect that 640x480 would still exist even if only the RGB camera is detected (here the minimum resolution is 960x540). If you set device ID to 1 or 2 in Preferences->Source, would you have a different output?

What is the version or your RealSenseViewer? Does your sensor have the latest firmware version? What is it? (RealSenseViewer->Info).

@rlabs-oss Can you launch rtabmap with Logger level set to Info to show those logs:

https://github.com/introlab/rtabmap/blob/80199f23b59c3982b70ebb3b2ad412067a2eca3e/corelib/src/camera/CameraRealSense2.cpp#L682-L689 . I would like to compare which video profiles are detected in Linux in comparison to Windows above. Also, what is the firmware installed on your L515? Thank you so far for this debugging!

EDIT: Looking at this post, it seems there is no 640x480 RGB profile... not sure how the current code is working on Linux?! Well, there they say 640x480 works?!

If i set the device id to 1 or 2 I get the following error:

grafik

I got this realsense viewer version: grafik

and this L500 firmware version: grafik

Anderson191919 commented 3 years ago

@matlabbe and @rlabs-oss thank you very much for your efforts. It helps me a lot. Maybe we can find the solution for this issue ^^

rlabs-oss commented 3 years ago

Just noticed librealsense is now V2.40.0 which may cause changes.

Using librealsense 2.39 on Windows 10 Pro, the L515 connected via USB3.2: RTAB-Map started and mapped as in Linux. -The default resolution is 848x480x60 - I changed that to 640x480x30 before starting.

log output is here

In device manager I see the L515 under USB devices and under Cameras Depth + RGB.

Ill check if this is related to the new V2.40 librealsense and find out if that changes the behavior.

Anderson191919 commented 3 years ago

Oh ok, I will install 2.39 librealsense. I will immediately give a response :)

Anderson191919 commented 3 years ago

grafik

This version, correct?

matlabbe commented 3 years ago

The current Windows build uses 2.39: https://github.com/introlab/rtabmap/blob/80199f23b59c3982b70ebb3b2ad412067a2eca3e/.appveyor.yml#L119 Maybe if you have upgraded to 2.40 and a new firmware update happened, this could have make the sensor incompatible with 2.39 used by rtabmap. I will upgrade to 2.40 on my linux and windows computer and see if my D435i and T265 still work ok.

Anderson191919 commented 3 years ago

ok thank you, I have tried 2.39.0.2337 but anyway the same error :/

Anderson191919 commented 3 years ago

Just noticed librealsense is now V2.40.0 which may cause changes.

Using librealsense 2.39 on Windows 10 Pro, the L515 connected via USB3.2: RTAB-Map started and mapped as in Linux. -The default resolution is 848x480x60 - I changed that to 640x480x30 before starting.

log output is here

In device manager I see the L515 under USB devices and under Cameras Depth + RGB.

Ill check if this is related to the new V2.40 librealsense and find out if that changes the behavior.

do you see the l515 as one device? like herre?

grafik

matlabbe commented 3 years ago

Is RealSenseViewer 2.39 working correctly with your device? (can you change resolution to 640x480 of the RGB camera?)

rlabs-oss commented 3 years ago

do you see the l515 as one device? like herre?

yes. In realsense-viewer @Anderson191919 you should see for the (/opps motion sensor) click the RGB slider to enable the RGB camera:- image

Anderson191919 commented 3 years ago

same here grafik

matlabbe commented 3 years ago

This is the depth camera, and for the RGB camera?

Anderson191919 commented 3 years ago

Is RealSenseViewer 2.39 working correctly with your device? (can you change resolution to 640x480 of the RGB camera?)

yes I can change to 640x480x30 and it works until I start the motion module, which kills the rgb stream. then I get some errors:

grafik grafik

Anderson191919 commented 3 years ago

This is the depth camera, and for the RGB camera?

grafik grafik

it is somehow not possible to change to 640x480 for the rgb stream

rlabs-oss commented 3 years ago

Should be, e.g image

@matlabbe Im in the process of build librealsense v2.40 and will also test that, not for this issue but generally over the cams I own. - The release notes recommend cmake .. -DFORCE_RSUSB_BACKEND=TRUE For V5 kernels.

matlabbe commented 3 years ago

I tried 2.40 on Ubuntu with D435i and T265 and it works as usual. Currently rebuilding Windows artifacts with 2.40, but I think it won't change anything, it is like the two cameras have not the same version, or that the firmwares are different. @Anderson191919 make sure you plug in the camera in a USB3 port, as long as you cannot have RGB 640x480 working in RealSenseViewer, it won't work in rtabmap either. You could try RealSenseViewer 2.40 to see if you can get the RGB 640x480 working.

rlabs-oss commented 3 years ago

I tried 2.40 on Ubuntu with D435i and T265 and it works as usual.

@matlabbe Rebuilt rtabmap + V2.40 librealsense: After camera firmware upgrades. The L515 works with global sync unchecked. The D455 reported a motion module failure on starting it and recovered (in realsense-viewer). - didn't affect rtabmap, The D455 is exceptionally fast with the madgwick filter enabled. D415 works however the IR mode inc depth IR looks quite underexposed.

matlabbe commented 3 years ago

Thank you @rlabs-oss! For the Global Sync issue, I think the original patch I received was from someone working on Windows, so the driver may work differently and Global Sync may work only on Windows (as you stated that kernel versions can affect this issue).

For D415 IR exposure, maybe there are some options available to change that. Check in RealSenseViewer for such options. You can then save the profile and load the presets in rtabmap with "D400 Series Visual Presets" option in Preferences->Source dialog. Some presets (look for best accuracy presets for rtabmap).

Anderson191919 commented 3 years ago

I tried 2.40 on Ubuntu with D435i and T265 and it works as usual. Currently rebuilding Windows artifacts with 2.40, but I think it won't change anything, it is like the two cameras have not the same version, or that the firmwares are different. @Anderson191919 make sure you plug in the camera in a USB3 port, as long as you cannot have RGB 640x480 working in RealSenseViewer, it won't work in rtabmap either. You could try RealSenseViewer 2.40 to see if you can get the RGB 640x480 working.

Have installed 2.40 but still no option of 640x480 for rgb stream :(

Anderson191919 commented 3 years ago

and plugged in usb 3

Anderson191919 commented 3 years ago

okkkkkkkkkeeeeyyyy i have solved it. thanks again for your help. maybe someone else have or will have the problem. here is the solution:

  1. download following firmware: 1.5.0.0 for the l515 (https://downloadcenter.intel.com/de/product/201775) grafik
  2. extract it
  3. run Intel.RealSense.Viewer.exe
  4. update the firmware by selecting the .bin-file grafik
  5. now you have the option to select 640x480x30 grafik
  6. start rtabmap (v0.20.6)
  7. under preferences... configure these settings: grafik
  8. enjoy your scan :)

again thanks for your help @matlabbe and @rlabs-oss 👍

rlabs-oss commented 3 years ago

@Anderson191919 Happy to read you have solved this. The realsense-viewer normally prompts to update firmware but perhaps there was something preventing this and nice to see the images how to do that which will help others with the same issue.

Anderson191919 commented 3 years ago

@Anderson191919 Happy to read you have solved this. The realsense-viewer normally prompts to update firmware but perhaps there was something preventing this and nice to see the images how to do that which will help others with the same issue.

I hope so :) the viewer prompts me with an update but the updated firmware does not work either. I have also tried version

All of them do not work for me but 1.5.0.0 works

matlabbe commented 3 years ago

Small note: for convenience, you don't have to explicitly set 640x480x30Hz, the resolution 848x480x60Hz parameters are ignored when L500 is detected, 640x480x30 is hard-coded: https://github.com/introlab/rtabmap/blob/7859313bebb988cc710703894eb457fdeb47c04a/corelib/src/camera/CameraRealSense2.cpp#L704-L707

When I will get my hands on a L515, I will check to add more resolution options and maybe also support IR-only mode (to test this demo on L515).

rlabs-oss commented 3 years ago

@Anderson191919 Happy to read you have solved this. The realsense-viewer normally prompts to update firmware but perhaps there was something preventing this and nice to see the images how to do that which will help others with the same issue.

I hope so :) the viewer prompts me with an update but the updated firmware does not work either. I have also tried version

* 1.4.1.0

* 1.4.1.2

* 1.5.1.3

* 1.5.2.0.

All of them do not work for me but 1.5.0.0 works

Does the update process show an error? Updating the firmware on the L515 does take time and appears to stop but eventually worked for me. Maybe check the USB port or try to confirm the port really is USB3.2 and also not via a hub. Original cable and try the rear motherboard ports if using a desktop. The speed at which the plug is inserted also matters its best to plug to the computer first then the camera.

Anderson191919 commented 3 years ago

During the firmware update the console showed some errors. Nevertheless the flash worked and got the corresponding firmware version. The firmware flashes with the other versions also work fine but are not showing 640x480x30 options in realsense.viewer. Therefore i used firmware 1.5.0.0

Maybe this also helps @samhoff20

Best regards ^^

shuixing85 commented 3 years ago

@matlabbe and @rlabs-oss thank you very much! It works like charm. I ve tried already some tools under ros, but the results with rtabmap standalone are already very impressive and much more stable! I will keep you posted about my progress. Cheers Peter

Hi Peta,

I have same error like [ERROR] (2020-10-20 20:04:53.761) CameraRealSense2.cpp:1374::captureImage() Missing frames (received 1)

The only difference is I'm running on win10,

can you please share your source code/commit to me?

thanks a lot, William

peta-peta commented 3 years ago

Hi William, this means that your camera is not set up properly within RTABmap. As explained above you need to untick the option Global time sync. See in the following picture: image

I am really interested on the accuracy improvement with synchronization. Are there any attempts to solve this issue?

sumitsarkar1 commented 3 years ago

this works fine after using this commit...I did the 3 hacks on Ubuntu 16 with L515...However I found only the last hack // if(isL500_) // desiredFramesetSize = 3; good enough to get L515 functional with RTabmap...However the 3D map created is not atall good due to some reason..I keep getting this warning CameraRealSense2.cpp:312::getPoseAndIMU() Could not find acc data to interpolate at image time 1606805391716.049561 after waiting 35 ms ...with all the 3 hacks too L515 is functional but the 3D map created is not good..it has multiple impressions of the same object as the lidar is moved slowly...looks like the 3D maps are not getting "stitched" togethar due to inaccurate motion estimation...any suggestions...

EDIT : this solved my problem So... 1... do the firmwire update 2....do the git checkout 3047b7d 3....do this hack only desiredFramesetSize = 2; //3 4....set 640 x 480 x 30Hz 5....uncheck everything except IR emitter enabled RtabMap should work with L515