search

minosworld / minos

MINOS: Multimodal Indoor Simulator
MIT License
201 stars 33 forks source link

[bug] Depth camera sees through walls #106

Open ducha-aiki opened 5 years ago

ducha-aiki commented 5 years ago

Under some conditions - bump into the wall and look diagonally, depth camera can see through walls. Look up and down are NOT used here.

How to reproduce:

python3 -m minos.tools.pygame_client --width 512 --height 512 --depth -s map --navmap --scene_ids 87f009ce2c904df0f0b75433611e74bf

And here is video of where to go:

https://drive.google.com/open?id=16Y6OSGX6HlCcktCYHaLPWyRwRNKhfxNE

ZhuFengdaaa commented 5 years ago

Meaningful discovery. This greatly harm the performance of model. Looking forward to bug fixing.

angelxuanchang commented 5 years ago

Thanks for reporting this. Did you observe this for other sensor types?

angelxuanchang commented 5 years ago

One thing you can try is in sensors.yml, setting the near for the depth to 0.001 (instead of the current 0.1 which is different from the other sensors).

ducha-aiki commented 5 years ago

@angelxuanchang I use only depth and RGB, so cannot say for other sensors. Regarding threshold - I will try, but it makes problem even worse, because I would like to use challenging conditions - "kinect" noise model, which clips depth at 0.5 > 0.1.

angelxuanchang commented 5 years ago

Please give it try and let us know if it fixes that issue and if it causes other problems. I think we set it higher for some reason (but I can't really remember why). For simulating kinect noise, it may make sense to clip afterwards.

ducha-aiki commented 5 years ago

@angelxuanchang yes, it helps! Thank you a lot!

angelxuanchang commented 5 years ago

That's great! Let us know if you see other problems. Also make sure to set the noise: true to use the simple depth noise model. Feel free to also add your own noise models. :-)

msavva commented 5 years ago

@ducha-aiki The issue you were observing is due to the larger default near plane setting (0.1) of the depth sensor relative to the color sensor (0.001). Thanks for reporting this -- it is indeed quite important to set the near plane distance to match. Cutting off for near and far depth distance thresholds can then be done in the noise model functions instead.