search

3dlg-hcvc / hssd

Code repository for the Habitat Synthetic Scenes Dataset (HSSD) paper.
https://3dlg-hcvc.github.io/hssd/
57 stars 5 forks source link

Previewing HSSD dataset scenes in Habitat-2.0 #2

Closed abhi1kumar closed 1 year ago

abhi1kumar commented 1 year ago

Hi HSSD team, Thank you for releasing a great dataset. I wanted to preview the HSSD scenes in Habitat-sim similar to the MP3D demo.

I arrange the MP3D scenes inside the habitat-sim as

habitat-sim
├── data
│      ├── scene_datasets
│      │      ├── mp3d
│      │      │     ├── 17DRP5sb8fy
│      │      │     ├── mp3d.scene_dataset_config.json
│      │      │     └── MP_TOS.pdf

When I run this demo from docs, it produces the RGB, instance seg and depth images (as expected).

import habitat_sim

import random
import matplotlib.pyplot as plt
import numpy as np

from utils import make_cfg, display_sample

test_scene    = "data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb"
scene_dataset = "data/scene_datasets/mp3d/mp3d.scene_dataset_config.json"
max_frames    = 2

sim_settings  = {
    "width": 256,  # Spatial resolution of the observations
    "height": 256,
    "scene": test_scene,  # Scene path
    "scene_dataset": scene_dataset,
    "default_agent": 0,
    "sensor_height": 1.5,  # Height of sensors in meters
    "color_sensor": True,  # RGB sensor
    "semantic_sensor": True,  # Semantic sensor
    "depth_sensor": True,  # Depth sensor
    "seed": 1,
}

cfg   = make_cfg(sim_settings)
sim   = habitat_sim.Simulator(cfg)
scene = sim.semantic_scene

random.seed(sim_settings["seed"])
sim.seed(sim_settings["seed"])

# Set agent state
agent       = sim.initialize_agent(sim_settings["default_agent"])
agent_state = habitat_sim.AgentState()
agent_state.position = np.array([0.0, 0.072447, 0.0])
agent.set_state(agent_state)

# Get agent state
agent_state = agent.get_state()
print("agent_state: position", agent_state.position, "rotation", agent_state.rotation)

total_frames = 0
action_names = list(
    cfg.agents[
        sim_settings["default_agent"]
    ].action_space.keys()
)

while total_frames < max_frames:
    action = random.choice(action_names)
    print("action", action)
    observations = sim.step(action)
    rgb = observations["color_sensor"]
    semantic = observations["semantic_sensor"]
    depth = observations["depth_sensor"]

    display_sample(rgb, semantic, depth, instance= True, show= True)

    total_frames += 1

Figure_1

I wanted a similar preview of HSSD scenes. I arranged the HSSD dataset from Huggingface, and placed them as follows:

habitat-sim
├── data
│      ├── hssd
│      │      ├── ai2thor-hab
│      │      ├── ai2thorhab-uncompressed
│      │      ├── hssd-hab
│      │      ├── hssd-models
│      │      ├── hssd-scenes
│      │
│      ├── scene_datasets
│      │      ├── mp3d

I then changed the values of variables as 

test_scene   = "data/hssd/hssd-scenes/scenes/102343992.glb" 
scene_dataset = "data/hssd/hssd-hab/hssd-hab.scene_dataset_config.json"

I then run the same demo file as

python demo.py

However, this time the code crashes with the following error message:

AssertionError: ESP_CHECK failed: Error loading general mesh data from file data/hssd/hssd-scenes/scenes/102343992.glb

How should one arrange the HSSD scenes inside the habitat-sim directory and what should be the values of test_scene and scene_dataset in the above demo file for previewing?

PS: My environment information is as follows: sshot

abhi1kumar commented 1 year ago

There was a downloading issue with the hssd-hab datasets from the huggingface and therefore, I redownloaded them with 

git lfs install
git lfs clone https://huggingface.co/datasets/hssd/hssd-hab

Next, I changed the following variables in the code:

test_scene    = "data/hssd/hssd-hab/scenes/102343992.scene_instance.json" 
scene_dataset = "data/hssd/hssd-hab/hssd-hab.scene_dataset_config.json"

I then run the same demo file as

python demo.py

And voila, the issue disappeared. Figure_1

abhi1kumar commented 1 year ago

Hi HSSD Team, I wanted to follow up on the RGB rendering of HSSD images. My images even in examples/viewer.py from habitat-sim look bad. hssd_mention

My rendered image is NOT good as the one below from the HSSD demo:

demo

I used the following script in habitat-sim for one of the scenes:

python examples/viewer.py --dataset data/hssd/hssd-hab/hssd-hab.scene_dataset_config.json --scene data/hssd/hssd-hab/scenes/102343992.scene_instance.json

It would be great if you could let us know how to obtain nice HSSD demo images on our end. e.g., do we need to adjust lightings?

PS - The example from matterport3D dataset data/scene_datasets/mp3d/17DRP5sb8fy/17DRP5sb8fy.glb looks OK in the habitat-sim viewer.

SamMaoYS commented 1 year ago

Hi @abhi1kumar, thank you for your interest in the HSSD dataset.

We used Blender's Eevee renderer to render the video, with uncompressed HSSD scene glb assets, as Blender currently does not support glb files with BasisU compressed textures as inputs. As for the blender rendering properties, we turned on Ambient Occlusion, Screen Space Reflections, Motion blur (for the video), and set High Contrast in Color Management. We also used interior.exr in Blender as the environment map and disabled shadows for the lighting.

In addition, the relatively recent versions of habitat-sim can produce significantly higher-fidelity rendering by setting appropriate configuration parameters to enable PBR materials and shading.

Best regards, Yongsen Mao

abhi1kumar commented 1 year ago

Thank you @SamMaoYS for your reply.

In addition, the relatively recent versions of habitat-sim can produce significantly higher-fidelity rendering by setting appropriate configuration parameters to enable PBR materials and shading.

I looked at the lighting tutorial. However, this code does not contain you are referring to. I also could not find anything on the HSSD-200 repo which relates to the PBR rendering.

I am very new to the embodied AI field and I might have missed relevant tutorials. Therefore, it would be amazing if you could share the link to a tutorial or share a small working code with configuration files to enable PBR materials and shading with a HSSD-200 scene, and the corresponding rendered result.

PS: I am using habitat-sim 0.2.4 and habitat-lab 0.2.4. Please also confirm if PBR rendering is feasible in this version.

mukulkhanna commented 1 year ago

Hello @abhi1kumar!

There have been a few significant improvements to Habitat's rendering in the latest versions. These have not been documented through tutorials, however.

For improved PBR and IBL (image-based lighting) rendering, you might want to install the latest habitat-sim and habitat-lab code from their main branches. Once you have done that, you can run the sim viewer with the --ibl flag as such:

./build/viewer --dataset /path/to/hssd-hab.scene_dataset_config.json --ibl -- 107734449_176000403

To have better rendering through a sim instance initialized using habitat-lab, you need to enable the IBL flag as such:

Add pbr_image_based_lighting: bool = True to this line in your habitat-lab installation.

abhi1kumar commented 1 year ago

Thank you @mukulkhanna for replying.

For improved PBR and IBL (image-based lighting) rendering, you might want to install the latest habitat-sim and habitat-lab code from their main branches. Once you have done that, you can run the sim viewer with the --ibl flag

I typed the following command in my original environment:

python examples/viewer.py --dataset data/hssd/hssd-hab/hssd-hab.scene_dataset_config.json --scene data/hssd/hssd-hab/scenes/102343992.scene_instance.json --ibl

And the result sshot

This is much better compared to what we got without the --ibl flag.

Add pbr_image_based_lighting: bool = True to this line in your habitat-lab installation.

Next, I made a new environment the latest habitat-sim and habitat-lab and then added this line before installing habitat-lab

I typed the following command in my new environment:

python examples/viewer.py --dataset data/hssd/hssd-hab/hssd-hab.scene_dataset_config.json --scene data/hssd/hssd-hab/scenes/102343992.scene_instance.json --ibl

The results look similar to the last one: sshot_3

Could you please confirm if you land up with something similar on your end.

mukulkhanna commented 1 year ago

@abhi1kumar, the pbr_image_based_lighting: bool = False change is for when you create a sim instance using a Python script using habitat. The habitat-sim viewer you are using (in both examples) does not use the above change in any way. When using the habitat-sim viewer, just passing the --ibl flag should be enough.

Also, if you were able to pass the --ibl flag and see improvements in your original environment, I think your habitat installations were already using the main branch (and not through conda or anything). If that is true, you can just stick to the original environment and maybe do a git pull to make sure you have the latest changes (if any since you cloned).

abhi1kumar commented 1 year ago

Thank you @mukulkhanna for your informative reply. I had a small request. Could you please screenshot and share the output of running the following command:

python examples/viewer.py --dataset data/hssd/hssd-hab/hssd-hab.scene_dataset_config.json --scene data/hssd/hssd-hab/scenes/102343992.scene_instance.json --ibl

in this thread. That would confirm that I am running correctly on my end.

mukulkhanna commented 1 year ago

If you built habitat-sim from the latest main branch, there is no reason for our outputs to look different, but here you go.

image

Also, in the command you shared above, just specifying the scene ID (instead of the scene_instance json path) should be enough:

python examples/viewer.py --dataset data/hssd/hssd-hab/hssd-hab.scene_dataset_config.json --scene 102343992 --ibl
abhi1kumar commented 1 year ago

Thank you @mukulkhanna for another quick reply. Your screenshot confirms that I am correctly rendering the HSSD-200 images.

I am closing this issue for now, and will get back to you if I face any more issues.

chengzu-li commented 8 months ago

Hi! I'm trying to preview the HSSD dataset scenes with habitat-lab (0.2.5) and habitat-sim (0.2.5) following the demo.py. However, when I try to instantiate the simulator: sim = habitat_sim.Simulator(cfg) the code crashes with the following error message, while it doesn't report any error when I view the scenes of MP3D. 

>>> sim = habitat_sim.Simulator(cfg)
Renderer: NVIDIA TITAN X (Pascal)/PCIe/SSE2 by NVIDIA Corporation
OpenGL version: 4.6.0 NVIDIA 525.147.05
Using optional features:
    GL_ARB_vertex_array_object
    GL_ARB_separate_shader_objects
    GL_ARB_robustness
    GL_ARB_texture_storage
    GL_ARB_texture_view
    GL_ARB_framebuffer_no_attachments
    GL_ARB_invalidate_subdata
    GL_ARB_texture_storage_multisample
    GL_ARB_multi_bind
    GL_ARB_direct_state_access
    GL_ARB_get_texture_sub_image
    GL_ARB_texture_filter_anisotropic
    GL_KHR_debug
    GL_KHR_parallel_shader_compile
    GL_NV_depth_buffer_float
Using driver workarounds:
    no-forward-compatible-core-context
    nv-egl-incorrect-gl11-function-pointers
    no-layout-qualifiers-on-old-glsl
    nv-zero-context-profile-mask
    nv-implementation-color-read-format-dsa-broken
    nv-cubemap-inconsistent-compressed-image-size
    nv-cubemap-broken-full-compressed-image-query
    nv-compressed-block-size-in-bits
[22:00:06:793734]:[Error]:[Metadata] StageAttributesManager.cpp(87)::registerObjectFinalize : Render asset template handle `stages/102343992` specified in stage template with handle :stages/102343992does not correspond to any existing file or primitive render asset, so StageAttributes registration is aborted.
[22:00:06:794251]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `1efdc3d37dfab1eb9f99117bb84c59003d684811` specified in object template with handle : `1efdc3d37dfab1eb9f99117bb84c59003d684811` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:794330]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `1efdc3d37dfab1eb9f99117bb84c59003d684811` specified in object template with handle : `1efdc3d37dfab1eb9f99117bb84c59003d684811` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:794640]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `356ce92bc38493578fdf63b7f3edfaea8001c849` specified in object template with handle : `356ce92bc38493578fdf63b7f3edfaea8001c849` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:794963]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `27dd125228e0872193eb34907b03ef9caf98289f` specified in object template with handle : `27dd125228e0872193eb34907b03ef9caf98289f` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:795037]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `27dd125228e0872193eb34907b03ef9caf98289f` specified in object template with handle : `27dd125228e0872193eb34907b03ef9caf98289f` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:795394]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `b4197540cbbe2787185cd37ab9f1524b883cbdbe` specified in object template with handle : `b4197540cbbe2787185cd37ab9f1524b883cbdbe` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:795468]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `b4197540cbbe2787185cd37ab9f1524b883cbdbe` specified in object template with handle : `b4197540cbbe2787185cd37ab9f1524b883cbdbe` does not correspond to any existing file or primitive render asset, so registration is aborted.
......
[22:00:06:842435]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` specified in object template with handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:842489]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` specified in object template with handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:842538]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` specified in object template with handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:842807]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `3753-0` specified in object template with handle : `3753-0` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:842847]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `3753-0` specified in object template with handle : `3753-0` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:843072]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `204-0` specified in object template with handle : `204-0` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:843297]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `220-0` specified in object template with handle : `220-0` does not correspond to any existing file or primitive render asset, so registration is aborted.
......
[22:00:06:851867]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `3753-3` specified in object template with handle : `3753-3` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:852086]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `218-18` specified in object template with handle : `218-18` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:852332]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `218-19` specified in object template with handle : `218-19` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:853185]:[Warning]:[Metadata] MetadataMediator.cpp(489)::getFilePathForHandle : <getSemanticSceneDescriptorPathByHandle> : Unable to find file path for  , so returning empty string.
[22:00:06:855653]:[Error]:[Core] ManagedContainerBase.h(330)::checkExistsWithMessage : <Stage Template>::getObjectCopyByHandle:Stage Template managed object handle `` not found in ManagedContainer, so aborting.

My file structure is: 

habitat-lab
├── data
│      ├── scene_datasets
│      │      ├── hssd-hab
│      │      │     ├── hssd-hab.scene_dataset_config.json
│      │      │     ├── hssd-hab-uncluttered.scene_dataset_config.json
│      │      │     └── objects
│      │      │      │.            └── ...
│      │      │     └── scenes
│      │      │      │.            └── ...
│      │      │     └── stages
│      │      │      │.            └── ...

Attached please also find my code. 

import habitat_sim
import numpy as np
import matplotlib.pyplot as plt
import random
import os

test_scene    = "./data/scene_datasets/hssd-hab/scenes/102343992.scene_instance.json" 
scene_dataset = "./data/scene_datasets/hssd-hab/hssd-hab.scene_dataset_config.json"

rgb_sensor = True  # @param {type:"boolean"}
depth_sensor = True  # @param {type:"boolean"}
semantic_sensor = True  # @param {type:"boolean"}

sim_settings = {
    "width": 256,  # Spatial resolution of the observations
    "height": 256,
    "scene": test_scene,  # Scene path
    "scene_dataset": scene_dataset,
    "default_agent": 0,
    "sensor_height": 1.5,  # Height of sensors in meters
    "color_sensor": rgb_sensor,  # RGB sensor
    "depth_sensor": depth_sensor,  # Depth sensor
    "semantic_sensor": semantic_sensor,  # Semantic sensor
    "seed": 1,  # used in the random navigation
    "enable_physics": False,  # kinematics only
}

def make_cfg(settings):
    sim_cfg = habitat_sim.SimulatorConfiguration()
    sim_cfg.gpu_device_id = 0
    sim_cfg.scene_id = settings["scene"]
    sim_cfg.enable_physics = settings["enable_physics"]
    # Note: all sensors must have the same resolution
    sensors = {
        "color_sensor": {
            "sensor_type": habitat_sim.SensorType.COLOR,
            "resolution": [settings["height"], settings["width"]],
            "position": [0.0, settings["sensor_height"], 0.0],
        },
        "depth_sensor": {
            "sensor_type": habitat_sim.SensorType.DEPTH,
            "resolution": [settings["height"], settings["width"]],
            "position": [0.0, settings["sensor_height"], 0.0],
        },
        "semantic_sensor": {
            "sensor_type": habitat_sim.SensorType.SEMANTIC,
            "resolution": [settings["height"], settings["width"]],
            "position": [0.0, settings["sensor_height"], 0.0],
        },
    }
    sensor_specs = []
    for sensor_uuid, sensor_params in sensors.items():
        if settings[sensor_uuid]:
            sensor_spec = habitat_sim.CameraSensorSpec()
            sensor_spec.uuid = sensor_uuid
            sensor_spec.sensor_type = sensor_params["sensor_type"]
            sensor_spec.resolution = sensor_params["resolution"]
            sensor_spec.position = sensor_params["position"]
            sensor_specs.append(sensor_spec)
    # Here you can specify the amount of displacement in a forward action and the turn angle
    agent_cfg = habitat_sim.agent.AgentConfiguration()
    agent_cfg.sensor_specifications = sensor_specs
    agent_cfg.action_space = {
        "move_forward": habitat_sim.agent.ActionSpec(
            "move_forward", habitat_sim.agent.ActuationSpec(amount=0.25)
        ),
        "turn_left": habitat_sim.agent.ActionSpec(
            "turn_left", habitat_sim.agent.ActuationSpec(amount=30.0)
        ),
        "turn_right": habitat_sim.agent.ActionSpec(
            "turn_right", habitat_sim.agent.ActuationSpec(amount=30.0)
        ),
    }
    return habitat_sim.Configuration(sim_cfg, [agent_cfg])

cfg = make_cfg(sim_settings)
sim = habitat_sim.Simulator(cfg)

May I know if you have any idea on why this error occurs? Thanks!

mukulkhanna commented 7 months ago

Hello,

The issue most likely is with the scene ID that has been passed. Having provided the path to the scene dataset config JSON, now you only need to provide only the scene ID (102343992) and not the path to the scene instance. That is, you need to change:

test_scene = "102343992"

The demo was made with MP3D in mind and does not directly translate to HSSD because of the difference in the nature of the datasets (static 3D reconstructed scenes vs synthetic scenes with dynamic assets) and how they are structured.

Please let me know if you face any more issues.

On Wed, Jan 24, 2024 at 2:13 PM Chengzu @.***> wrote:

Hi! I'm trying to preview the HSSD dataset scenes with habitat-lab (0.2.5) and habitat-sim (0.2.5) following the demo.py. However, when I try to instantiate the simulator: sim = habitat_sim.Simulator(cfg) the code crashes with the following error message, while it doesn't report any error when I view the scenes of MP3D.

sim = habitat_sim.Simulator(cfg) Renderer: NVIDIA TITAN X (Pascal)/PCIe/SSE2 by NVIDIA Corporation OpenGL version: 4.6.0 NVIDIA 525.147.05 Using optional features: GL_ARB_vertex_array_object GL_ARB_separate_shader_objects GL_ARB_robustness GL_ARB_texture_storage GL_ARB_texture_view GL_ARB_framebuffer_no_attachments GL_ARB_invalidate_subdata GL_ARB_texture_storage_multisample GL_ARB_multi_bind GL_ARB_direct_state_access GL_ARB_get_texture_sub_image GL_ARB_texture_filter_anisotropic GL_KHR_debug GL_KHR_parallel_shader_compile GL_NV_depth_buffer_float Using driver workarounds: no-forward-compatible-core-context nv-egl-incorrect-gl11-function-pointers no-layout-qualifiers-on-old-glsl nv-zero-context-profile-mask nv-implementation-color-read-format-dsa-broken nv-cubemap-inconsistent-compressed-image-size nv-cubemap-broken-full-compressed-image-query nv-compressed-block-size-in-bits [22:00:06:793734]:[Error]:[Metadata] StageAttributesManager.cpp(87)::registerObjectFinalize : Render asset template handle stages/102343992 specified in stage template with handle :stages/102343992does not correspond to any existing file or primitive render asset, so StageAttributes registration is aborted. [22:00:06:794251]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 1efdc3d37dfab1eb9f99117bb84c59003d684811 specified in object template with handle : 1efdc3d37dfab1eb9f99117bb84c59003d684811 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:794330]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 1efdc3d37dfab1eb9f99117bb84c59003d684811 specified in object template with handle : 1efdc3d37dfab1eb9f99117bb84c59003d684811 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:794640]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 356ce92bc38493578fdf63b7f3edfaea8001c849 specified in object template with handle : 356ce92bc38493578fdf63b7f3edfaea8001c849 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:794963]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 27dd125228e0872193eb34907b03ef9caf98289f specified in object template with handle : 27dd125228e0872193eb34907b03ef9caf98289f does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:795037]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 27dd125228e0872193eb34907b03ef9caf98289f specified in object template with handle : 27dd125228e0872193eb34907b03ef9caf98289f does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:795394]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : b4197540cbbe2787185cd37ab9f1524b883cbdbe specified in object template with handle : b4197540cbbe2787185cd37ab9f1524b883cbdbe does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:795468]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : b4197540cbbe2787185cd37ab9f1524b883cbdbe specified in object template with handle : b4197540cbbe2787185cd37ab9f1524b883cbdbe does not correspond to any existing file or primitive render asset, so registration is aborted. ...... [22:00:06:842435]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 9c61e7e98634fda17cd2276857cea3c03ad1024d specified in object template with handle : 9c61e7e98634fda17cd2276857cea3c03ad1024d does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:842489]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 9c61e7e98634fda17cd2276857cea3c03ad1024d specified in object template with handle : 9c61e7e98634fda17cd2276857cea3c03ad1024d does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:842538]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 9c61e7e98634fda17cd2276857cea3c03ad1024d specified in object template with handle : 9c61e7e98634fda17cd2276857cea3c03ad1024d does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:842807]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 3753-0 specified in object template with handle : 3753-0 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:842847]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 3753-0 specified in object template with handle : 3753-0 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:843072]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 204-0 specified in object template with handle : 204-0 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:843297]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 220-0 specified in object template with handle : 220-0 does not correspond to any existing file or primitive render asset, so registration is aborted. ...... [22:00:06:851867]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 3753-3 specified in object template with handle : 3753-3 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:852086]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 218-18 specified in object template with handle : 218-18 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:852332]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : 218-19 specified in object template with handle : 218-19 does not correspond to any existing file or primitive render asset, so registration is aborted. [22:00:06:853185]:[Warning]:[Metadata] MetadataMediator.cpp(489)::getFilePathForHandle : : Unable to find file path for , so returning empty string. [22:00:06:855653]:[Error]:[Core] ManagedContainerBase.h(330)::checkExistsWithMessage : ::getObjectCopyByHandle:Stage Template managed object handle `` not found in ManagedContainer, so aborting.

My file structure is:

habitat-lab ├── data │ ├── scene_datasets │ │ ├── hssd │ │ │ ├── hssd-hab.scene_dataset_config.json │ │ │ ├── hssd-hab-uncluttered.scene_dataset_config.json │ │ │ └── objects │ │ │ │. └── ... │ │ │ └── scenes │ │ │ │. └── ... │ │ │ └── stages │ │ │ │. └── ...

Attached please also find my code.

import habitat_sim import numpy as np import matplotlib.pyplot as plt import random import os

test_scene = "./data/scene_datasets/hssd-hab/scenes/102343992.scene_instance.json" scene_dataset = "./data/scene_datasets/hssd-hab/hssd-hab.scene_dataset_config.json"

rgb_sensor = True # @param {type:"boolean"} depth_sensor = True # @param {type:"boolean"} semantic_sensor = True # @param {type:"boolean"}

sim_settings = { "width": 256, # Spatial resolution of the observations "height": 256, "scene": test_scene, # Scene path "scene_dataset": scene_dataset, "default_agent": 0, "sensor_height": 1.5, # Height of sensors in meters "color_sensor": rgb_sensor, # RGB sensor "depth_sensor": depth_sensor, # Depth sensor "semantic_sensor": semantic_sensor, # Semantic sensor "seed": 1, # used in the random navigation "enable_physics": False, # kinematics only }

def make_cfg(settings): sim_cfg = habitat_sim.SimulatorConfiguration() sim_cfg.gpu_device_id = 0 sim_cfg.scene_id = settings["scene"] sim_cfg.enable_physics = settings["enable_physics"]

Note: all sensors must have the same resolution

sensors = {
    "color_sensor": {
        "sensor_type": habitat_sim.SensorType.COLOR,
        "resolution": [settings["height"], settings["width"]],
        "position": [0.0, settings["sensor_height"], 0.0],
    },
    "depth_sensor": {
        "sensor_type": habitat_sim.SensorType.DEPTH,
        "resolution": [settings["height"], settings["width"]],
        "position": [0.0, settings["sensor_height"], 0.0],
    },
    "semantic_sensor": {
        "sensor_type": habitat_sim.SensorType.SEMANTIC,
        "resolution": [settings["height"], settings["width"]],
        "position": [0.0, settings["sensor_height"], 0.0],
    },
}
sensor_specs = []
for sensor_uuid, sensor_params in sensors.items():
    if settings[sensor_uuid]:
        sensor_spec = habitat_sim.CameraSensorSpec()
        sensor_spec.uuid = sensor_uuid
        sensor_spec.sensor_type = sensor_params["sensor_type"]
        sensor_spec.resolution = sensor_params["resolution"]
        sensor_spec.position = sensor_params["position"]
        sensor_specs.append(sensor_spec)
# Here you can specify the amount of displacement in a forward action and the turn angle
agent_cfg = habitat_sim.agent.AgentConfiguration()
agent_cfg.sensor_specifications = sensor_specs
agent_cfg.action_space = {
    "move_forward": habitat_sim.agent.ActionSpec(
        "move_forward", habitat_sim.agent.ActuationSpec(amount=0.25)
    ),
    "turn_left": habitat_sim.agent.ActionSpec(
        "turn_left", habitat_sim.agent.ActuationSpec(amount=30.0)
    ),
    "turn_right": habitat_sim.agent.ActionSpec(
        "turn_right", habitat_sim.agent.ActuationSpec(amount=30.0)
    ),
}
return habitat_sim.Configuration(sim_cfg, [agent_cfg])

cfg = make_cfg(sim_settings) sim = habitat_sim.Simulator(cfg)

May I know if you have any idea on why this error occurs? Thanks!

— Reply to this email directly, view it on GitHub https://github.com/3dlg-hcvc/hssd/issues/2#issuecomment-1909000831, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF5SBUXKCUYH6MYYMPUGU7LYQGBQRAVCNFSM6AAAAAAZ5BYPJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBZGAYDAOBTGE . You are receiving this because you were mentioned.Message ID: @.***>

Eugleo commented 6 months ago

@mukulkhanna Hey, the lightning fix from earlier doesn't seem to work anymore, --ibl is not a known argument for the viewer. What is the new way to achieve better lightning?

sxshco commented 6 months ago

@chengzu-li The problem should be that you forget to set the scene_dataset parameter in make_cfg. You can add it to:

def make_cfg(settings):
    sim_cfg = habitat_sim.SimulatorConfiguration()
    sim_cfg.gpu_device_id = 0
    sim_cfg.scene_id = settings["scene"]
    sim_cfg.enable_physics = settings["enable_physics"]

    if "scene_dataset" in settings:
        sim_cfg.scene_dataset_config_file = settings["scene_dataset"]

Hi! I'm trying to preview the HSSD dataset scenes with habitat-lab (0.2.5) and habitat-sim (0.2.5) following the demo.py. However, when I try to instantiate the simulator: sim = habitat_sim.Simulator(cfg) the code crashes with the following error message, while it doesn't report any error when I view the scenes of MP3D.

>>> sim = habitat_sim.Simulator(cfg)
Renderer: NVIDIA TITAN X (Pascal)/PCIe/SSE2 by NVIDIA Corporation
OpenGL version: 4.6.0 NVIDIA 525.147.05
Using optional features:
    GL_ARB_vertex_array_object
    GL_ARB_separate_shader_objects
    GL_ARB_robustness
    GL_ARB_texture_storage
    GL_ARB_texture_view
    GL_ARB_framebuffer_no_attachments
    GL_ARB_invalidate_subdata
    GL_ARB_texture_storage_multisample
    GL_ARB_multi_bind
    GL_ARB_direct_state_access
    GL_ARB_get_texture_sub_image
    GL_ARB_texture_filter_anisotropic
    GL_KHR_debug
    GL_KHR_parallel_shader_compile
    GL_NV_depth_buffer_float
Using driver workarounds:
    no-forward-compatible-core-context
    nv-egl-incorrect-gl11-function-pointers
    no-layout-qualifiers-on-old-glsl
    nv-zero-context-profile-mask
    nv-implementation-color-read-format-dsa-broken
    nv-cubemap-inconsistent-compressed-image-size
    nv-cubemap-broken-full-compressed-image-query
    nv-compressed-block-size-in-bits
[22:00:06:793734]:[Error]:[Metadata] StageAttributesManager.cpp(87)::registerObjectFinalize : Render asset template handle `stages/102343992` specified in stage template with handle :stages/102343992does not correspond to any existing file or primitive render asset, so StageAttributes registration is aborted.
[22:00:06:794251]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `1efdc3d37dfab1eb9f99117bb84c59003d684811` specified in object template with handle : `1efdc3d37dfab1eb9f99117bb84c59003d684811` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:794330]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `1efdc3d37dfab1eb9f99117bb84c59003d684811` specified in object template with handle : `1efdc3d37dfab1eb9f99117bb84c59003d684811` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:794640]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `356ce92bc38493578fdf63b7f3edfaea8001c849` specified in object template with handle : `356ce92bc38493578fdf63b7f3edfaea8001c849` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:794963]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `27dd125228e0872193eb34907b03ef9caf98289f` specified in object template with handle : `27dd125228e0872193eb34907b03ef9caf98289f` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:795037]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `27dd125228e0872193eb34907b03ef9caf98289f` specified in object template with handle : `27dd125228e0872193eb34907b03ef9caf98289f` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:795394]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `b4197540cbbe2787185cd37ab9f1524b883cbdbe` specified in object template with handle : `b4197540cbbe2787185cd37ab9f1524b883cbdbe` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:795468]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `b4197540cbbe2787185cd37ab9f1524b883cbdbe` specified in object template with handle : `b4197540cbbe2787185cd37ab9f1524b883cbdbe` does not correspond to any existing file or primitive render asset, so registration is aborted.
......
[22:00:06:842435]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` specified in object template with handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:842489]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` specified in object template with handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:842538]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` specified in object template with handle : `9c61e7e98634fda17cd2276857cea3c03ad1024d` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:842807]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `3753-0` specified in object template with handle : `3753-0` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:842847]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `3753-0` specified in object template with handle : `3753-0` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:843072]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `204-0` specified in object template with handle : `204-0` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:843297]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `220-0` specified in object template with handle : `220-0` does not correspond to any existing file or primitive render asset, so registration is aborted.
......
[22:00:06:851867]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `3753-3` specified in object template with handle : `3753-3` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:852086]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `218-18` specified in object template with handle : `218-18` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:852332]:[Error]:[Metadata] ObjectAttributesManager.cpp(258)::registerObjectFinalize : Render asset template handle : `218-19` specified in object template with handle : `218-19` does not correspond to any existing file or primitive render asset, so registration is aborted.
[22:00:06:853185]:[Warning]:[Metadata] MetadataMediator.cpp(489)::getFilePathForHandle : <getSemanticSceneDescriptorPathByHandle> : Unable to find file path for  , so returning empty string.
[22:00:06:855653]:[Error]:[Core] ManagedContainerBase.h(330)::checkExistsWithMessage : <Stage Template>::getObjectCopyByHandle:Stage Template managed object handle `` not found in ManagedContainer, so aborting.

My file structure is:

habitat-lab
├── data
│      ├── scene_datasets
│      │      ├── hssd-hab
│      │      │     ├── hssd-hab.scene_dataset_config.json
│      │      │     ├── hssd-hab-uncluttered.scene_dataset_config.json
│      │      │     └── objects
│      │      │      │.            └── ...
│      │      │     └── scenes
│      │      │      │.            └── ...
│      │      │     └── stages
│      │      │      │.            └── ...

Attached please also find my code.

import habitat_sim
import numpy as np
import matplotlib.pyplot as plt
import random
import os

test_scene    = "./data/scene_datasets/hssd-hab/scenes/102343992.scene_instance.json" 
scene_dataset = "./data/scene_datasets/hssd-hab/hssd-hab.scene_dataset_config.json"

rgb_sensor = True  # @param {type:"boolean"}
depth_sensor = True  # @param {type:"boolean"}
semantic_sensor = True  # @param {type:"boolean"}

sim_settings = {
    "width": 256,  # Spatial resolution of the observations
    "height": 256,
    "scene": test_scene,  # Scene path
    "scene_dataset": scene_dataset,
    "default_agent": 0,
    "sensor_height": 1.5,  # Height of sensors in meters
    "color_sensor": rgb_sensor,  # RGB sensor
    "depth_sensor": depth_sensor,  # Depth sensor
    "semantic_sensor": semantic_sensor,  # Semantic sensor
    "seed": 1,  # used in the random navigation
    "enable_physics": False,  # kinematics only
}

def make_cfg(settings):
    sim_cfg = habitat_sim.SimulatorConfiguration()
    sim_cfg.gpu_device_id = 0
    sim_cfg.scene_id = settings["scene"]
    sim_cfg.enable_physics = settings["enable_physics"]
    # Note: all sensors must have the same resolution
    sensors = {
        "color_sensor": {
            "sensor_type": habitat_sim.SensorType.COLOR,
            "resolution": [settings["height"], settings["width"]],
            "position": [0.0, settings["sensor_height"], 0.0],
        },
        "depth_sensor": {
            "sensor_type": habitat_sim.SensorType.DEPTH,
            "resolution": [settings["height"], settings["width"]],
            "position": [0.0, settings["sensor_height"], 0.0],
        },
        "semantic_sensor": {
            "sensor_type": habitat_sim.SensorType.SEMANTIC,
            "resolution": [settings["height"], settings["width"]],
            "position": [0.0, settings["sensor_height"], 0.0],
        },
    }
    sensor_specs = []
    for sensor_uuid, sensor_params in sensors.items():
        if settings[sensor_uuid]:
            sensor_spec = habitat_sim.CameraSensorSpec()
            sensor_spec.uuid = sensor_uuid
            sensor_spec.sensor_type = sensor_params["sensor_type"]
            sensor_spec.resolution = sensor_params["resolution"]
            sensor_spec.position = sensor_params["position"]
            sensor_specs.append(sensor_spec)
    # Here you can specify the amount of displacement in a forward action and the turn angle
    agent_cfg = habitat_sim.agent.AgentConfiguration()
    agent_cfg.sensor_specifications = sensor_specs
    agent_cfg.action_space = {
        "move_forward": habitat_sim.agent.ActionSpec(
            "move_forward", habitat_sim.agent.ActuationSpec(amount=0.25)
        ),
        "turn_left": habitat_sim.agent.ActionSpec(
            "turn_left", habitat_sim.agent.ActuationSpec(amount=30.0)
        ),
        "turn_right": habitat_sim.agent.ActionSpec(
            "turn_right", habitat_sim.agent.ActuationSpec(amount=30.0)
        ),
    }
    return habitat_sim.Configuration(sim_cfg, [agent_cfg])

cfg = make_cfg(sim_settings)
sim = habitat_sim.Simulator(cfg)

May I know if you have any idea on why this error occurs? Thanks!