search

HpWang-whu / RoReg

[TPAMI 2023] RoReg: Pairwise Point Cloud Registration with Oriented Descriptors and Local Rotations
https://hpwang-whu.github.io/RoReg/
90 stars 12 forks source link

Point clouds frame #4

Closed fdila closed 7 months ago

fdila commented 8 months ago

Hi, I have a doubt about using RoReg on custom data. Must the point cloud be expressed in the sensor's reference frame (i.e. with the center in 0,0,0) due to the rotations that have to be applied for extracting the features? I'm having some problem with using the pretrained network and trying to align point clouds that are expressed w.r.t. a "map" frame, and I was wondering if that's the issue.

Thanks

HpWang-whu commented 7 months ago

Hi @fdila , Thanks for your interest and sorry for the late reply! RoReg feature does not require the frame to be expressed in the sensor's reference and you can optionally centralize the frame. To generalize pre-trained RoReg(YOHO)-Desc on your own data, please refer to here.

Yours,

fdila commented 7 months ago

I asked this question because we are trying to align pointclouds from the ETH dataset (the same sequences as your original experiments). Different from your experiments, before feeding the pointclouds to the neural network, we apply an initial misalignment to the source point cloud. Moreover, we are using pointclouds which are expressed in the "map frame". Under these conditions RoReg almost always outputs an identity matrix.

Does your answer imply that the network should be retrained under these conditions (point clouds expressed not in the sensor frame and with an initial misalignment applied)?

HpWang-whu commented 7 months ago

Hi @fdila, I have tried RoReg under your settings and it can perform well without re-training. I think the problem might be that the keypoints saved are the ones before 'mapping' and thus fail to correctly extract keypoint features. You can solve it by changing this line https://github.com/HpWang-whu/RoReg/blob/d599cda3163b1601fb8d416259ff338cbed12860/dataops/dataset.py#L110 to if True:.

Moreover, delete data/YOHO_FCGF fold before running because RoReg uses the saved features rather than re-extracting them by default.

Yours,

fdila commented 7 months ago

I had modified the code to automatically extract the keypoints instead of loading them from a .txt file and probably introduced some bug while doing it. I solved it and RoReg now works as expected, thank you!