Utilising Semantic Region Support Implemented in Release 0.3.1

[frank-gallagher]

Habitat-Sim version

v0.3.1

❓ Questions and Help

Hi @aclegg3,

Release 0.3.1 added support for Semantic Regions. In Particular, _habitat_sim.scene.SemanticScene.get_regions_forpoint(), _habitat_sim.scene.SemanticRegion.poly_looppoints, _habitat_sim.scene.SemanticRegion.floorheight and _habitat_sim.scene.SemanticRegion.extrusion height_.

How, and to which configs, do I add the associated data for the HM3D semantics dataset to be able to use the above?

[aclegg3]

Hey @frank-gallagher,

Sorry the documentation isn't available yet. I'll share a quick example here which could unblock you. The following are modifications to the existing example hm3d scene dataset for scene GLAQ4DNUx5U:

First, add the following example region annotation as a new file: hm3d/example/00861-GLAQ4DNUx5U/GLAQ4DNUx5U.semantic_config.json:

{
  "region_annotations": [
    {
      "name": "gym",
      "label": "workout/gym/exercise",
      "poly_loop": [
        [
          0.26291,
          0.0,
          -9.53374
        ],
        [
          -3.285,
          0.0,
          -9.53374
        ],
        [
          -3.286,
          0.0,
          -13.758
        ],
        [
          0.26391,
          0.0,
          -13.758
        ]
      ],
      "floor_height": 0.0,
      "extrusion_height": 2.8,
      "min_bounds": [
        -3.286,
        0.0,
        -13.758
      ],
      "max_bounds": [
        0.26391,
        2.8,
        -9.53374
      ]
    }
}

Next, add the following dummy scene instance config as a new file: hm3d/example/00861-GLAQ4DNUx5U/GLAQ4DNUx5U.scene_instance.json:

{
  "stage_instance": {
        "template_name": "GLAQ4DNUx5U"
    }
}

Then modify the scene_dataset file hm3d/example/hm3d_annotated_example_basis.scene_dataset_config.json which pulls these all together:

{
  "stages": {
    "paths": {
      ".glb": [
        "00861-GLAQ4DNUx5U/*.basis.glb"
      ]
    },
    "default_attributes": {
      "shader_type": "flat",
      "up": [
        0,
        0,
        1
      ],
      "front": [
        0,
        1,
        0
      ],
      "origin": [
        0,
        0,
        0
      ],
      "has_semantic_textures": true
    }
  },
  "objects": {},
  "light_setups": {},
  "scene_instances": {
    "default_attributes": {
      "default_lighting": "no_lights",
      "semantic_scene_instance": "%%CONFIG_NAME_AS_ASSET_FILENAME%%.semantic_config.json"
    },
    "paths": {
      ".json": [
        "00861-GLAQ4DNUx5U/*.scene_instance.json"
      ]
    }
  },
  "semantic_scene_descriptor_instances": {
    "default_attributes": {
      "semantic_descriptor_filename": "%%CONFIG_NAME_AS_ASSET_FILENAME%%.semantic.txt",
      "semantic_asset": "%%CONFIG_NAME_AS_ASSET_FILENAME%%.semantic.glb"
    },
    "paths": {
      ".json": [
        "00861-GLAQ4DNUx5U/*.semantic_config.json"
      ]
    }
  }
}

Note that I moved the semantic defaults into the new semantic_scene_descriptor_instance defaults.

Now when I run the viewer:

python examples/viewer.py --dataset data/scene_datasets/hm3d/example/hm3d_annotated_example_basis.scene_dataset_config.json --scene GLAQ4DNUx5U

The regions are loaded and can be visualized with 'j' or printed by injecting the following after initialization:

print(f"Regions = {len(self.sim.semantic_scene.regions)}")
for region in self.sim.semantic_scene.regions:
    print(f"Region: {region.id}")

[frank-gallagher]

Hi @aclegg3 ,

That's great, thanks very much for the example.