Add more instructions?

[c7w]

Thanks for your nice piece of work! However, when trying to reproduce the results you reported in your paper, I enconutered some issues.

I can successfully reproduce the results on the SUN RGBD benchmark and the KITTI benchmark. However, when trying to reproduce results for the ScanNet benchmark, using the projects/configs/uni3detr/uni3detr_scannet_large.py as the config file, I harvest results with metrics equalling to almost 0 when inferring with your provided checkpoint. Also I tried to train with this config myself and harvest results really low after the training ended.

+----------------+---------+---------+---------+---------+
| classes        | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet        | 0.0081  | 0.4543  | 0.0001  | 0.0672  |
| bed            | 0.1152  | 0.8765  | 0.0098  | 0.2840  |
| chair          | 0.0653  | 0.7829  | 0.0067  | 0.2639  |
| sofa           | 0.0524  | 0.7835  | 0.0023  | 0.2062  |
| table          | 0.0683  | 0.6457  | 0.0071  | 0.1371  |
| door           | 0.0047  | 0.3640  | 0.0002  | 0.0792  |
| window         | 0.0115  | 0.3298  | 0.0002  | 0.0284  |
| bookshelf      | 0.0258  | 0.5974  | 0.0029  | 0.1688  |
| picture        | 0.0002  | 0.0676  | 0.0000  | 0.0000  |
| counter        | 0.0268  | 0.4808  | 0.0000  | 0.0192  |
| desk           | 0.0698  | 0.8819  | 0.0039  | 0.2441  |
| curtain        | 0.0103  | 0.4478  | 0.0000  | 0.0149  |
| refrigerator   | 0.0131  | 0.5439  | 0.0040  | 0.1579  |
| toilet         | 0.0309  | 0.7241  | 0.0050  | 0.3103  |
| sink           | 0.0039  | 0.3878  | 0.0000  | 0.0306  |
| bathtub        | 0.2044  | 0.6452  | 0.0823  | 0.1935  |
| garbagebin     | 0.0026  | 0.3887  | 0.0000  | 0.0547  |
| showercurtrain | 0.0104  | 0.4286  | 0.0000  | 0.0000  |
+----------------+---------+---------+---------+---------+
| Overall        | 0.0402  | 0.5461  | 0.0069  | 0.1256  |
+----------------+---------+---------+---------+---------+

I processed the ScanNet dataset following the instruction of mmdet3d. Could you please kindly provide the training logs for the ScanNet benchmark, as well as the training config file for the S3DIS benchmark? Thanks in advance.

---

Enviroment:

torch                     1.13.1+cu116
mmcv-full                 1.5.2
mmdet                     2.28.2
mmdet3d                   1.3.0       /DATA_EDS2/gaoha/workspace/2023/Uni3DETR (modified from v1.0.0rc5)
mmengine                  0.9.0
mmsegmentation            0.28.0

[zhenyuw16]

We will add related instructions soon. For ScanNet, the input point clouds need global alignment https://github.com/open-mmlab/mmdetection3d/blob/v1.0.0rc5/mmdet3d/datasets/pipelines/transforms_3d.py#605. The original mmdetection3d put it in the config file, but here we pre-process point clouds first with global alignment. You can add this to obtain the correct results

[CodeWZT]

@c7w hi, do you try the solution provide by @zhenyuw16 ？is it work?

[Fantasy314]

Thanks for your nice piece of work! However, when trying to reproduce the results you reported in your paper, I enconutered some issues.

I can successfully reproduce the results on the SUN RGBD benchmark and the KITTI benchmark. However, when trying to reproduce results for the ScanNet benchmark, using the projects/configs/uni3detr/uni3detr_scannet_large.py as the config file, I harvest results with metrics equalling to almost 0 when inferring with your provided checkpoint. Also I tried to train with this config myself and harvest results really low after the training ended.

+----------------+---------+---------+---------+---------+
| classes        | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet        | 0.0081  | 0.4543  | 0.0001  | 0.0672  |
| bed            | 0.1152  | 0.8765  | 0.0098  | 0.2840  |
| chair          | 0.0653  | 0.7829  | 0.0067  | 0.2639  |
| sofa           | 0.0524  | 0.7835  | 0.0023  | 0.2062  |
| table          | 0.0683  | 0.6457  | 0.0071  | 0.1371  |
| door           | 0.0047  | 0.3640  | 0.0002  | 0.0792  |
| window         | 0.0115  | 0.3298  | 0.0002  | 0.0284  |
| bookshelf      | 0.0258  | 0.5974  | 0.0029  | 0.1688  |
| picture        | 0.0002  | 0.0676  | 0.0000  | 0.0000  |
| counter        | 0.0268  | 0.4808  | 0.0000  | 0.0192  |
| desk           | 0.0698  | 0.8819  | 0.0039  | 0.2441  |
| curtain        | 0.0103  | 0.4478  | 0.0000  | 0.0149  |
| refrigerator   | 0.0131  | 0.5439  | 0.0040  | 0.1579  |
| toilet         | 0.0309  | 0.7241  | 0.0050  | 0.3103  |
| sink           | 0.0039  | 0.3878  | 0.0000  | 0.0306  |
| bathtub        | 0.2044  | 0.6452  | 0.0823  | 0.1935  |
| garbagebin     | 0.0026  | 0.3887  | 0.0000  | 0.0547  |
| showercurtrain | 0.0104  | 0.4286  | 0.0000  | 0.0000  |
+----------------+---------+---------+---------+---------+
| Overall        | 0.0402  | 0.5461  | 0.0069  | 0.1256  |
+----------------+---------+---------+---------+---------+

I processed the ScanNet dataset following the instruction of mmdet3d. Could you please kindly provide the training logs for the ScanNet benchmark, as well as the training config file for the S3DIS benchmark? Thanks in advance.

Enviroment:

torch                     1.13.1+cu116
mmcv-full                 1.5.2
mmdet                     2.28.2
mmdet3d                   1.3.0       /DATA_EDS2/gaoha/workspace/2023/Uni3DETR (modified from v1.0.0rc5)
mmengine                  0.9.0
mmsegmentation            0.28.0

Hello, I would like to ask you that I can't reproduce the experiment according to your configuration, in which the MMCV is not compatible, how can this be solved? Below is my runtime command, can I ask for your advice please? conda create --name b python=3.8 -y conda activate b conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia pip install -U openmim -i https://pypi.tuna.tsinghua.edu.cn/simple mim install mmengine==0.9.0 -i https://pypi.tuna.tsinghua.edu.cn/simple pip install mmcv_full-1.5.2-cp38-cp38-manylinux1_x86_64.whl mim install mmdet==2.28.2 -i https://pypi.tuna.tsinghua.edu.cn/simple pip install -v -e . python demo/pcd_demo.py demo/data/kitti/000008.bin pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth --show