search

verlab / 3DSemanticMapping_JINT_2020

Repository for the paper "Extending Maps with Semantic and Contextual Object Information for Robot Navigation: a Learning-Based Framework using Visual and Depth Cues"
MIT License
89 stars 17 forks source link

Using ZED2 stereo camera instead of Kinect #2

Open esmaeilinia opened 4 years ago

esmaeilinia commented 4 years ago

How can I run this using a stereo camera (ZED2), and without turtlebot? Im trying to modify RTAB-Map to read from a stereo camera, but obj_positioner fails to position the detected objects on the map. I have modified the nodes to read from ZED stereo camera, but still it cant work with RTAB-map. Can you please provide more details on using different RGB-D cameras other than Kinect? Thanks

germal commented 3 years ago

Hi @esmaeilinia , I have exactly the same issue with a d435 camera, all the detected objects (i.e. person,book) are positioned at the origin of the map frame, even if I have adapted the camera frame and the pointcloud topic in the object_positioner.yaml file and in the object_projector.cpp. Darknet_ros publish correctly the coordinates of the target objects. Did you manage to resolve this issue ? @renatojmsdh Renato could you please help to understand ? Thanks for the support germal

germal commented 3 years ago

Hello @renatojmsdh @ericksonrn @ericksonrn @michelmelosilva could you please help to understand the root cause of the issue ? Is this a misconfiguration of mine ? Thanks Regards

esmaeilinia commented 3 years ago

Hi @germal, I couldn't resolve it. I had the same issue with ZED and D435i.

renatojmsdh commented 3 years ago

Hello @esmaeilinia and @germal,

I will have a look to check what is happening. Could you please tell me:

1) What is your ROS distro (Kinetic, Melodic, ...)?

2) Does the object_loalizer/filter node show any error message? (roslaunch auto obj_positioner.launch)

3) [IMPORTANT] Could you please record and share with me one small ROS bag file from the D435i and one from ZED that you are trying to use?

Thanks

germal commented 3 years ago

Hello @renatojmsdh Thank you for your reply. I have ROS Melodic . There aren't error messages ; looking at the code , it seems to me that object_marker.py and map_interceptor.py are parametrized on the "door" object . The door object class doesn't appear in my darknet yaml file standard file ( I didn't custom train) , could be this the reason ? I will record a rosbag asap Thanks for your support ! Regards

esmaeilinia commented 3 years ago

@renatojmsdh Thanks for your reply. I have tested on both Kinetic and Melodic. There was no error message. I'll share a sample ROS bag asap. Many thanks for your support.

germal commented 3 years ago

Hi @renatojmsdh I have uploaded the bag with the following topics /d400/color/image_rect_color /d400/depth/image_rect_raw /d400/color/camera_info /d400/depth_registered/points /tf /tf_static /darknet_ros/bounding_boxes /object_positioner/raw_marker https://wetransfer.com/downloads/05aa8e39f3272f46d91cafb54776969120210128150139/4b676bfaf22a56eff8aedf7ad727e33420210128150218/ca7a59

Thanks for your help ! image

 

germal commented 3 years ago

Hello @renatojmsdh , Please let us know if we can provide further details for your analysis. Thanks Regards germal

renatojmsdh commented 3 years ago

Hi @germal, I took your bag and I will likely check it this weekend. I keep you informed soon. Best

germal commented 3 years ago

Hello @renatojmsdh , @ericksonrn , @michelmelosilva After a careful analysis of my colleagues more expert than me , seems that there is no issue indeed, your reasearch code works only with your bag and this is fine. I think you should amend ,for the sake of clarity,at least your readme,remarking that the results can be achieved only offline ,with your bag file, and not with a real camera. Thanks Regards

imdsafi09 commented 4 months ago

Hello @renatojmsdh Any update regarding that ZED2 bag files?