search

csiro-robotics / InCloud

[IROS2022] Official repository of InCloud: Incremental Learning for Point Cloud Place Recognition, Published in IROS2022 https://arxiv.org/abs/2203.00807
https://ieeexplore.ieee.org/document/9981252
Other
36 stars 9 forks source link

How can I implement LOGG3D correctly? #3

Open cloudcjf opened 1 year ago

cloudcjf commented 1 year ago

Dear authors,

Thanks for your exciting work. I am reproducing your results and have noticed that logg3d is not yet implemented. So I take the official code of logg3d to forward the global embeddings(256-dim vectors) and use the default config as the code you open source. Everything seems just fine since the code is successfully running. However, the recall@1 and recall@1% on Oxford and In-house datasets are too low to believe. Do you have any ideas on this issue? You have mentioned in your paper that oxford and In-house lack insufficient overlap between adjacent point clouds. Any influence on the training of logg3d? Hope you can help me solve this issue at your convenience.

JBKnights commented 1 year ago

Hi CloudCJF. The main challenge in forwarding LoGG3D as implemented in the official repository is in applying the point-based loss, as a) the overlap between clouds is significantly smaller for the Oxford and In-House datasets than for MulRan and KITTI and b) the ground truth poses for Oxford and In-House are only the XY positions of each cloud, which is not sufficient to align the clouds for a point-based loss.

In order to implement LoGG3D in our paper we make the following changes:

cloudcjf commented 1 year ago

Hi CloudCJF. The main challenge in forwarding LoGG3D as implemented in the official repository is in applying the point-based loss, as a) the overlap between clouds is significantly smaller for the Oxford and In-House datasets than for MulRan and KITTI and b) the ground truth poses for Oxford and In-House are only the XY positions of each cloud, which is not sufficient to align the clouds for a point-based loss.

In order to implement LoGG3D in our paper we make the following changes:

  • We disable the point-based loss, as it is not possible to align the clouds in the Oxford and In-House datasets accurately enough with available GT to do so
  • We deploy the backbone and SoP into the training pipeline used for MinkLoc3D (i.e. using the same data pre-processing, hard negative mining in-batch ,etc.)

  • We use the following hyperparameters:

    • Incremental loss weight: 100
    • Batch Size Limit: 200

Hopefully this helps, please let us know if there's any other questions you have!

Hi, @JBKnights . Thanks for your explanation. Actually, I have already disabled the point-based loss and used the sparcify_and_collate_list function in dataset_utils.py to transfer the point clouds into the sparse tensor batch. Then the logg3d model is called just like minkloc3d and PointNetVLAD in the main loop of trainer.py. This will return a batch of embedding vectors(size:[batch, 256]). Regarding the hyperparameters, I have followed the default you give. hyperparameters of incremental learning are not used yet since I'd like to train logg3d from scratch first. After 60 epochs of training, the recall@1 on Oxford and In-house is less than 10%. There must be something important I missed. Do you have any idea about what leads to the wrong result?

peymmo commented 1 year ago

@kavisha725 you may able to help @cloudcjf here. Thank you.

kavisha725 commented 1 year ago

For the general guidelines on how to use LoGG3D-Net please go through the readme page at: https://github.com/csiro-robotics/LoGG3D-Net and/or create an issue in that repo.

Unfortunately I'm not familiar with how LoGG3D-Net was used in the InCloud project. The only suggestions I can make are the general ml debugging steps of observing the loss and validation curves during training. For more specific information on how LoGG3D-Net was used in InCloud, it's best to wait till @JBKnights replies.

cloudcjf commented 1 year ago

For the general guidelines on how to use LoGG3D-Net please go through the readme page at: https://github.com/csiro-robotics/LoGG3D-Net and/or create an issue in that repo.

Unfortunately I'm not familiar with how LoGG3D-Net was used in the InCloud project. The only suggestions I can make are the general ml debugging steps of observing the loss and validation curves during training. For more specific information on how LoGG3D-Net was used in InCloud, it's best to wait till @JBKnights replies.

Thank you for your kind reply.@kavisha725 Actually, I have tried to replace SOP with max-pooling or NetVLAD to implement LoGG3D-Net in the InCloud and both are working. This may suggest that the original SOP is not suitable for the datasets in the InCloud project. Hope @JBKnights can help me find the solution. Thanks in advance.

peymmo commented 4 months ago

@cloudcjf is there anything we can help you with?