How to use RGBD Images as input?

CMU-TBD / SocNavBench

A Grounded Simulation Testing Framework for Evaluating Social Navigation: https://arxiv.org/abs/2103.00047

MIT License

34 stars 6 forks source link

How to use RGBD Images as input? #23

Open mphamhung opened 2 years ago

mphamhung commented 2 years ago

I've set render_mode to full-render as in: https://github.com/CMU-TBD/SocNavBench/issues/9#issuecomment-885727674

But I'm not sure how to get and use the RGBD data during the robot's "sense" step.

ajdroid commented 2 years ago

@mphamhung a patch is coming with some instructions on how to do this in SocNavBench, but it is fairly slow to use the rgbd image in the sensing loop. If you really need to use RGBD, one place to look is SEAN which is a Unity based simulator we have recently been working with to update with the SocNavBench pedestrian replay method and scenes.