Any plans to reintegrate the 2d bbox data collection?

[sibojia]

I noticed in the commit history there was a branch implementing 2d bbox collection calculated by 3d bbox and camera intrinsic parameters, but was later reverted. I wonder the reason of removing the function, is it because the code breaks the plugin (as in #41 ) or the result is not right? Is there a plan to reintegrate this feature? @ai-tor @dtmoodie

Thanks a lot!

[dtmoodie]

Hi,

My assumption is that there are issues with occlusion detection causing false positives. One researcher found cases where GTA's collision detection did not work correctly and thus 2d bounding boxes were drawn for detections that were occluded by other vehicles / buildings / etc. We discussed that a fix for that may be to use ray tracing to do a higher quality validation on the 2d bounding box.

I also didn't get the notification that I created a bug that was causing crashes in #41 . I'll have to revisit that.

[sibojia]

Thanks for your reply. I used to generate 2d detection data from GTA with Renderdoc (a graphics debugging tool). That tool is able to get a stencil mask of the current frame which is roughly a semantic segmentation mask, so I was able to find out whether the car is occluded by checking the pixels in the mask. I don't know if there exists similar methods in the mod environment.

[IanKirwan]

@sibojia GTAVisionExport can be used for both 2D and 3D bbox capture. HTH