search

leolyj / DCA-SRSFE

This is the code related to "Deformation and Correspondence Aware Unsupervised Synthetic-to-Real Scene Flow Estimation for Point Clouds" (CVPR 2022).
MIT License
37 stars 6 forks source link

Reproduction of paper results #5

Open alexbgl opened 1 year ago

alexbgl commented 1 year ago

Hi @leolyj,

thank you for sharing code and data of your very interesting work!

When trying to reproduce the results from your paper, I encountered the following two issues:

1) for GTA-SF ---> KITTI-SF, I managed to roughly reproduce your results, both when just doing inference with your provided checkpoints and when training the adaptation beginning from the checkpoint of the baseline model. Thus, both data pre-processing and training routine seem to work here. However, I noticed that training and validation data from the KITTI dataset are identical. Is that correct or did I miss something?

2) for GTA-SF ---> Lyft-SF, I get roughly the same results as in the paper when doing inference with your provided checkpoints. Thus, data pre-processing and seems to be correct here as well. However, when starting the adaptation process from the checkpoint of the baseline model on my own, I could not reach the performance reported in the paper. Compared to an EPE3D of 0.1277, I achieve around 0.135 before the model converges/begins to slightly overfit (please see the attached training curve). I also noted that the model achieved top performance already after around 2800 iterations while you trained for around 46.000 iterations (15 epochs * 12430 data samples / 4 batch size). Do you have any idea what the reasons could be? Are there possibly any hyper-parameters I need to adjust?

Thanks in advance! grafik

JZ-9962 commented 1 year ago

Hi, have you modified the EPS parameter when adapting to different target dataset? In our experiments, EPS is set to 0.6 for waymo, 0.4 for kitti and 0.3 for lyft. You can try to adjust this hyper-parameter. Good luck!