search

XuyangBai / D3Feat

[TensorFlow] Official implementation of CVPR'20 oral paper - D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features https://arxiv.org/abs/2003.03164
MIT License
261 stars 38 forks source link

Performance of FCGF on ETH dataset #1

Closed ShengyuH closed 4 years ago

ShengyuH commented 4 years ago

hi Xuyang,

Can you confirm the performance of FCGF on ETH dataset as reported in your paper? It's significantly worse than PerfectMatch and is a bit strange to me.

Best, Shengyu

XuyangBai commented 4 years ago

Hi @ShengyuH I agree that the FCGF is a powerful local descriptor for 3D point clouds, but in this experiment, we are testing the generalization ability of the models and the reason why FCGF didn't perform well is in the following.

During the experiment, I find the receptive field is even more important than the voxel size because ETH datasets are in a large spatial range and low resolution. For PerfectMatch, they use a larger value of W for the SDV voxel grid (from 0.3m to 1m) when adapting the weight trained on 3DMatch to ETH, which enlarges the receptive field of the local patches. And for our method, we are also able to increase the receptive field of each point without changing the voxel size (by scaling up the grid size of each layer). But for FCGF, I didn't find a good way to increase the receptive field while fixing the voxel size, thus I directly use pre-trained weight of 5cm voxel size on 3DMatch to test on the ETH dataset. I think that's the main reason why the performance of FCGF on ETH is worse.

Best, Xuyang.

ShengyuH commented 4 years ago

hi @XuyangBai Thanks for your detailed explanation. If I calculate correctly, FCGF has a tensor stride of 8, together with many local convolution operations and 0.05m voxel resolution, we can actually get a bigger receptive field than PerfectMatch/D3Feat. I ignore the fact that 3DMatch and ETH dataset have different resolutions, I think this could explain the poor generalisation ability, as kernels are not normalised to fit such domain gap.

Best, Shengyu

chrischoy commented 4 years ago

Hi @ShengyuH and @XuyangBai,

Indeed, FCGF requires the same metric spaces for training and testing, which is more than often the case. However, this seems to be the failure mode for the ETH dataset.

Great work! Chris

XuyangBai commented 4 years ago

Hi @chrischoy

Thanks for your explanation!

Best, Xuyang