[QUESTION] Support for Habitat / mp3d dataset

[troxlepa]

Hello Nathan, Thanks for the great work on Hydra! I've noticed that there exists a launch file and config for the Matterport3d dataset. The corresponding rViz config shows the subscription to the habitat data (e.g./habitat/agent_0/pointcloud) Is there existing work on exporting data from habitat to Hydra?

[nathanhhughes]

Hi, thanks for your interest in our work! Yes, we've done some work with habitat (and specifically Matterport3d within habitat), but unfortunately the interfaces that we have to habitat aren't really ready for release yet and I'm not entirely sure if or when we'll release them.

It's somewhat straightforward to create a similar interface to the one we have if you're on ubuntu 20.04. We publish the following information from habitat:

• RGB image (for debug purposes)
• Depth image (with a 32FC1 encoding)
• Semantic image (we collapse the instance labels from habitat into category labels using the semantic_scene.objects field in the simulator object, which defines a map for a given instance c between instance label (c.id) and category label (c.category.index())
• TF of the current agent pose

You do also have to be careful about the left-handed habitat coordinate system in habitat. Also note that our actual interface is a little more complicated (we use zmq as a bridge between habitat and the ROS node publishing the sensor information to avoid python issues on ubuntu 18.04).