Sometimes, images will be labelled with the wrong bounding box. This seems to be a timing issue since the (randomly jumping) camera has already moved by the time the image has rendered (sometimes we can see a black image or a half-rendered image being generated). This issue makes it so we have to do manual cleaning on data which can be tedious.
I see a few possible paths to solve this issue:
1) Make sure the image rendering in Gazebo can be synchronized with the publishing of bounding box data (perhaps through timestamps in ROS message headers). This may not be possible if Gazebo doesn't expose the internal renderer so we can force a render at a specific time.
2) Gazebo wasn't designed for objects jumping around discontinuously. We can modify the random camera plugin so that instead of jumping, the camera moves smoothly along a trajectory toward the next random pose.
3) Create our own version of Gazebo's camera plugin that does everything (render rgb, render depth, get bounding box) all at once to avoid timing issues.
Option 3 seems perhaps the most future-proof since it solves the core issue rather than the symptoms.
Sometimes, images will be labelled with the wrong bounding box. This seems to be a timing issue since the (randomly jumping) camera has already moved by the time the image has rendered (sometimes we can see a black image or a half-rendered image being generated). This issue makes it so we have to do manual cleaning on data which can be tedious.
I see a few possible paths to solve this issue: 1) Make sure the image rendering in Gazebo can be synchronized with the publishing of bounding box data (perhaps through timestamps in ROS message headers). This may not be possible if Gazebo doesn't expose the internal renderer so we can force a render at a specific time. 2) Gazebo wasn't designed for objects jumping around discontinuously. We can modify the random camera plugin so that instead of jumping, the camera moves smoothly along a trajectory toward the next random pose. 3) Create our own version of Gazebo's camera plugin that does everything (render rgb, render depth, get bounding box) all at once to avoid timing issues.
Option 3 seems perhaps the most future-proof since it solves the core issue rather than the symptoms.