allenai / Holodeck

CVPR 2024: Language Guided Generation of 3D Embodied AI Environments.
https://yueyang1996.github.io/holodeck
Apache License 2.0
304 stars 25 forks source link

Floor reachable points abnormality after importing to ai2thor #25

Closed huluwulu1115 closed 3 months ago

huluwulu1115 commented 3 months ago

I generated some scenes using Holodeck, but I found there is a possibility(about 30%) that the Floor reachable points are incorrectly generated, like the one in screen shot. The reachable points are obtained through controller.step(action="GetReachablePositions").metadata["actionReturn"].

I also set the gridsize to 0.25 while initialing the controller, as follows:

controller = Controller(
    agentMode="default",
    renderDepthImage=renderDepthImage,
    renderInstanceSegmentation=renderInstanceSegmentation,
    renderSemanticSegmentation=renderSemanticSegmentation,
    renderNormalsImage=renderNormalsImage,      
    makeAgentsVisible=False,
    visibilityDistance=1.5,
    scene=scene,
    gridSize=0.25,
    width=1024,
    height=1024,
    fieldOfView=90,
    action_hook_runner=ProceduralAssetHookRunner(
        asset_directory=objaverse_dir,
        asset_symlink=True,
        verbose=True,
    ),
)

I wonder if that is an innate problem of ai2thor, or that is something caused by holodeck.

image
YueYANG1996 commented 3 months ago

@Lucaweihs, could you please help with this?

Lucaweihs commented 3 months ago

Hi @huluwulu1115 - can you provide the full script you used to generate those points along with the house json file? The y values there do suggest that there is something weird going on. Perhaps you can try my response here (see also the responses below) to ensure the agent is in a collision free starting position after initializing the scene.