search

ARISE-Initiative / robosuite

robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
https://robosuite.ai
Other
1.37k stars 429 forks source link

Depth normalization is unhelpful when depth is infinite #368

Open jbonyun opened 1 year ago

jbonyun commented 1 year ago

We had an environment in which we had no walls/background, so our depth camera was seeing our scene, and was seeing infinite darkness. Robosuite is normalizing the depth so that the full range of 8-bits is used. This sounds helpful, but when you have infinity in the mix, it squashes everything else into one value, giving no distinction between depths of the relevant scene.

We worked around by using get_real_depth_map to get the raw values, capping that at 6m (for our use case), and recalculating the normalized depth map. (Note that "infinity" was actually being reported as 145m, so there already is a cap, but it's really far.)

Maybe you could have an optional parameter for the maximum depth to be captured? Maybe infinity could be ignored in the normalization, and be given whatever the next-furthest value is? Maybe the 255 value could be reserved for infinity, and normalize everything else into 0-254? Or maybe you are happy not supporting infinite depth.

zhuyifengzju commented 1 year ago

Thanks for the feedback! We will take this corner case into consideration and update this function.