search

nickgkan / butd_detr

Code for the ECCV22 paper "Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Clouds"
Other
74 stars 11 forks source link

shift on the detected boxes #2

Closed clorislili closed 1 year ago

clorislili commented 1 year ago

Hi, I visualize the detected boxes (before random shift augmentation) on the point cloud scene, but realize that there is a consistent shift existed. After generating target box based on the point cloud scene, it resulted in 0.0 iou due to the shift. Would you plz public the detector backbone training code? Or any other ways to solve this problem? Thanks in advance!

ayushjain1144 commented 1 year ago

Hi @clorislili, just to get more information, are you facing this issue with the predictions we saved or you are training a detector and saving the predictions yourself?

clorislili commented 1 year ago

@ayushjain1144 Thanks for your reply. Yes, with the predicted boxes you offered. Specifically, I visualized the all_detected_boxes after 590 line in joint_det_dataset.py before any augmentation. And the point cloud scene I use followed referit3d codebase, I believe there is no shift option in the point cloud scene. When I visualize detected boxes and point cloud scene together, it can not overlap, some shift existed between boxes and point cloud scene. Maybe the way you deal with pointc cloud scene is different from referit3d?

clorislili commented 1 year ago

658d9a18d842af64cdba37be Can you see the shift of the boxes and the corresponding point cloud object?

ayushjain1144 commented 1 year ago

Yep! Clearly it looks incorrect. Which visualization code are you using? I am wondering because we haven't tested the visualization code in our dataloader for a long time, so maybe there is a bug there that we can look into.

Most likely though, I think the issue is related to using PC loaded by referit instead of our dataloader (we can look into it and get back to you on how you can fix that). Did you try visualizing the PC loaded by us and our detected boxes?

clorislili commented 1 year ago

The visualization code i use is yours, the wandb.3d visualizer. I think the problem does not lie here. I'm trying to use the PC by yours now, hope I can find where the problem is. Get back to me if you have got any progress. Thanks!

clorislili commented 1 year ago

Yeah, I figured it. It's because of the PC data. I will alter to your PC dataloader. Thanks a lot!

ayushjain1144 commented 1 year ago

Great that you figured it out! Closing this issue, but please feel free to ask any other questions you might have!

IISCAditayTripathi commented 1 year ago

What was the issue @ayushjain1144 and @clorislili ?

ayushjain1144 commented 1 year ago

Hi @IISCAditayTripathi, @clorislili debugged it themselves so I am not sure, but I believe the issue was visualizing the bounding boxes we provide with pointclouds from some other repo and there is likely some mismatch in coordinate system/alignment.