search

slothfulxtx / MBPTrack3D

[ICCV2023] MBPTrack: Improving 3D Point Cloud Tracking with Memory Networks and Box Priors
24 stars 6 forks source link

Reproduction of the result. #7

Open IzhiSu opened 1 year ago

IzhiSu commented 1 year ago

Dear authors,

I am not able to reproduce the results reported in the paper using the given config at config/mbptrack_kitti_ped_cfg.yaml.

I wonder if the same config used for generating the results in the paper is provided. Or is there anything else we can take note of the reproduce the results?

Thanks!

slothfulxtx commented 12 months ago

Hi, I've uploaded all my code and checkpoints on github, thus it should be easy to reproduce our results. I've just tested my code and checkpoint on our server

python main.py configs/mbptrack_kitti_ped_cfg.yaml --phase test --resume_from pretrained/mbptrack_kitti_ped.ckpt

The results are listed below image Notably, this server is also occupied by my colleagues for other tasks, thus the runtime metric is not accurate.

IzhiSu commented 12 months ago

Thanks for your reply! When I reproduced the category such as ’Pedestrian‘, the experimental results were 66.25/91.10. Could you please provide me with specific configurations to help me reproduce this result? Looking forward to your reply!

slothfulxtx commented 11 months ago

Actually, MBPTrack reuses previous prediction results when the tracked target is missing. If the prediction of MBPTrack is not accurate enough, it may lose targets for the following frames. (You can find some tracklets of extremely low quality in KITTI.) Thus the performance of MBPTrack is not very robust and has a minor fluctuation. I suggest you can train our model from scratch to reproduce the performance.