search

open-rmf / rmf_obstacle

Packages that infer the presence of obstacles from sensor inputs.
Apache License 2.0
5 stars 10 forks source link

Realsense D400 series Cameras support? #7

Open HappySamuel opened 1 year ago

HappySamuel commented 1 year ago

Hi

Can this rmf_obstacle_detectors work with any Realsense D400 series Cameras? If i wanna use D400 series cameras, what are the steps to integrate them up?

Best, Samuel

Yadunund commented 1 year ago

There are several modalities of obstacle detection. Apart from rmf_human_detector_oakd, the other packages are hardware agnostic and perform inference based on ROS 2 sensor msg data. The realsense-ros driver can help publish RGB images which you can pass to rmf_human_detector. You may even be able to convert the depth pt cloud into a laser scan msg and pass that data to rmf_obstacle_detector_laserscan.

If you're willing to contribute code, I can also guide you on enhancing rmf_human_detector to a) make it a more generalized detector (not just humans) b) Incorporate (optionally) a depth stream to obtain 3D pose data of the inferences. Right now some heuristics are used to estimate depth from 2D RGB image.

HappySamuel commented 1 year ago

Hi @Yadunund

That would be great to have rmf_human_detector to be a) more generalized detector b) incorporate with depth stream

I am willing to contribute the code under your guidance on achieving this.

Best, Samuel

Yadunund commented 1 year ago

Hi @HappySamuel sorry for the overdue response. I was occupied with other efforts. I should have more time for this towards the end of this month. Will put a plan together. Feel free to ping here again if you don't hear from me.

HappySamuel commented 1 year ago

Hi @Yadunund Good to hear from you again. Looking forward to work with you together on this.

Best, Samuel

Yadunund commented 5 months ago

@HappySamuel sorry for the long silence here.

Open-RMF is part of GSoC 2024 and one of the projects is to cleanup and improve the obstacle detection pipeline. You may want to checkout https://discourse.ros.org/t/attention-students-open-robotics-google-summer-of-code-2024-projects/

I've also opened #22 with a list of items to work on. The goal will be to support a wide range of sensors and detection algorithms via a plugin based system. Let me know if you're still keen on contributing.