search

facebookresearch / habitat-sim

A flexible, high-performance 3D simulator for Embodied AI research.
https://aihabitat.org/
MIT License
2.64k stars 424 forks source link

missing glass surfaces when using replica in habitat #2419

Open liren-jin opened 4 months ago

liren-jin commented 4 months ago

Hi there,

I am using replicav1 dataset in habitat_sim, python api. I can get correct images from the simulator, however, I realize that all glass surfaces are missing and simulator returns 0 depth in this case. Does anyone know how to adding the glass surfaces in the habitat simulator?

aclegg3 commented 4 months ago

Perhaps you can share some images of your expectation vs. reality? Often times glass can be a failure point in 3D scanning so I'm curious if you are seeing artifacts of that process or issues with the simulation/renderer.

liren-jin commented 4 months ago

Hi thanks for the reply.The original replica dataset has a mirror in this scene, but what i get from habitat is a missing surface on that area. I also check the mesh.ply file used by habitat-sim, it shows missing surfaces as well. so I guess because habitat-sim does not use the glass.sur file contained in the replica dataset?

replica dataset: Screenshot from 2024-06-29 10-31-07

image rendered from habitat-sim Screenshot from 2024-06-29 10-32-36

aclegg3 commented 4 months ago

Ah, I see. Yeah, that makes sense. We actually aren't rendering PTEX anymore, instead treating these as standard PLY. My guess is that this is one of the missed features.