[PyTorch] Official implementation of CVPR2022 paper "TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers". https://arxiv.org/abs/2203.11496
Hey @XuyangBai,
First of all, I would like to congratulate you on your great work. It is truly an awesome contribution!
I've been trying to apply TransFusion on the ViewOfDelft-Dataset (VOD). The data (I Use the radar data) set is in KITTI format. Therefore I have taken the configuration file from #51 and adapted it. My config now looks as follows:
Note that the original VOD point_cloud_range is [0, -25.6, -3, 51.2, 25.6, 2]. But if I change it accordingly, the model does not even converge. Might be relatet to #17 and #64 as the bbox_loss does not decrease. So my First Question is if you know how I can configure the loss (or architecture) so that I can run the original point_cloud_range.
The Radar Set of VOD provides an Nx7 array as Input. For simplicity I only used RCS in addition to the Points (Nx4).
However, the behavior I will now show does not change when I use Nx7. I also disabled augmentaion as some of the methods don't work with the Radar Data -> described in the VOD-Paper
I will try to explain my Second Question based on the log of my last training epoch. Full Log can be found here. Throughout the training (in all epochs) the loss decreases continuously. Also the matched_ious looks not to bad. However in the eval I only get zeros and some weird values. Is there something wrong with my config or do you have any Idea from where this weird behavoir originates frome?
Hey @XuyangBai, First of all, I would like to congratulate you on your great work. It is truly an awesome contribution! I've been trying to apply TransFusion on the ViewOfDelft-Dataset (VOD). The data (I Use the radar data) set is in KITTI format. Therefore I have taken the configuration file from #51 and adapted it. My config now looks as follows:
Note that the original VOD
point_cloud_range
is[0, -25.6, -3, 51.2, 25.6, 2]
. But if I change it accordingly, the model does not even converge. Might be relatet to #17 and #64 as the bbox_loss does not decrease. So my First Question is if you know how I can configure the loss (or architecture) so that I can run the originalpoint_cloud_range
. The Radar Set of VOD provides an Nx7 array as Input. For simplicity I only used RCS in addition to the Points (Nx4). However, the behavior I will now show does not change when I use Nx7. I also disabled augmentaion as some of the methods don't work with the Radar Data -> described in the VOD-PaperI will try to explain my Second Question based on the log of my last training epoch. Full Log can be found here. Throughout the training (in all epochs) the loss decreases continuously. Also the matched_ious looks not to bad. However in the eval I only get zeros and some weird values. Is there something wrong with my config or do you have any Idea from where this weird behavoir originates frome?
Thank you so much in advance! Best Leon