search

qcr / benchbot

BenchBot is a tool for seamlessly testing & evaluating semantic scene understanding tools in both realistic 3D simulation & on real robots
BSD 3-Clause "New" or "Revised" License
110 stars 12 forks source link

RGB and Depth images not aligned #66

Closed antabangun closed 2 years ago

antabangun commented 2 years ago

Hello!

Thank you for this wonderful simulator. I encountered an issue that might be related to #12 and #62.

I ran an agent with passive action commands and superimposed the extracted rgb and disparity (1/depth) images. But I think that the images don't align in many frames. The figure below illustrates this concern, showing "shadow" artifacts superimposed_1

And sometimes the difference is quite severe. superimposed_2

I'm using an RTX2080Ti. Is there any workaround for this issue? Thank you

david2611 commented 2 years ago

Hi @antabangun and thanks for your issue. Yes this does appear to be linked to both #12 and #62 and, as far as we can tell, is directly linked to the Omniverse simulator which seems to not always provide synchronized data. We had hoped that there would be sorted in the port to Omniverse but it does still seem to be an issue which is noticeable when the robot undergoes sharp angular movement which can be particularly prevalent in some steps of passive mode.

To "fix" this we have added a very short delay between when a move action has been completed and when it's observations can be sent through to the user. The delay is currently set to 0.6 seconds and is available on the develop branch (to be shortly pushed onto master). If you want to try it out before the master push just perform a benchbot_install -b develop to have your benchbot system synced up with the current develop branch for all repos.

If you continue to see this issue after using our fix, please don't hesitate to let us know :+1: