search

WHU-USI3DV / CoFiI2P

[IEEE RA-L 2024 & ICRA'25] CoFiI2P: Coarse-to-Fine Correspondences-Based Image-to-Point Cloud Registration
https://whu-usi3dv.github.io/CoFiI2P/
90 stars 5 forks source link

Question on Registration Recall #18

Closed qsisi closed 6 days ago

qsisi commented 1 month ago

Thanks for open-sourcing this project!

I have a question related to the registration recall on KITTI, looks like in your paper, the recall is 100% on kitti: image but when I ran the evaluation on kitti, the angles_diff and t_diff printed out is something like: image most of the printed information indicates the angle & translation diff are large, which is strange to me. Could you provide some hints about it?

Thanks for your reply!

martin-liao commented 1 month ago

Did you load the pre-trained weight correctly?

qsisi commented 1 month ago

I think I've loaded the pre-trained model correctly, I only modified this line: image and here are the results: image

qsisi commented 1 month ago

kitti train from scratch： image kitti finetuned on cofii2p_kitti.t7： image Looks like the released weight cofii2p_kitti only converges on loss_coarse?

martin-liao commented 1 month ago

I have tested the pre-trained model for the KITTI dataset. It seems like everything is fine. I will re-check the code for training and evaluation tomorrow. image

qsisi commented 1 month ago

Any updates over the last few days?

martin-liao commented 1 month ago

We have double-checked the evaluation code and everything is fine. Could you please provide more details, like configurations?

martin-liao commented 1 month ago

kitti train from scratch： image kitti finetuned on cofii2p_kitti.t7： image Looks like the released weight cofii2p_kitti only converges on loss_coarse? We have made further attempts and found that the evaluation results (RRE & RTE) are close to the values reported in the paper. We also recommend using Tensorboard to visualize the training details.

qsisi commented 1 month ago

After re-installing the cuda、torch、open3d as required, the problem is solved.

qsisi commented 1 month ago

Hello! Sorry to bother you again, but my question is what is the purpose to normalize the query features here?

https://github.com/WHU-USI3DV/CoFiI2P/blob/main/model/transformer/transformer.py#L53

Thanks for your reply.

martin-liao commented 4 weeks ago

It's just an implementation trick. We found that normalized query vector leads to better performance. Other variants may work, while you can have a try.