search

ethz-asl / hfnet

From Coarse to Fine: Robust Hierarchical Localization at Large Scale with HF-Net (https://arxiv.org/abs/1812.03506)
MIT License
776 stars 185 forks source link

When re-generating SfM via superpoint, the number of pointclouds is too little so it raise some unexpected errors! #42

Closed LimHyungTae closed 4 years ago

LimHyungTae commented 4 years ago

Hello, Mr. Sariln! :) First, thanks for giving these high-quality codes.

We want to employ your code on our environment, so we tested codes in the colmap-helpers on Aachen dataset before porting, yet it does not work. Specifically, we followed the colmap-helpers instruction that written on the README.md, so tried to re-construct SfM model via superpoint, but the SfM have little pointclouds (almost 80,000 3-D points) compared to the provided SfM model (almost 680,000 3-D points). So, I hope to get some information to generate SfM model from superpoint in detail to use it on the other dataset. Thanks in advance!

sarlinpe commented 4 years ago

Hi, thank you for you interest. It is difficult for me to provide support with so little information. I know several people that successfully used the provided scripts to run SfM without much problems. I recommend rerunning the reconstruction again and paying attention to any error messages. I will release in the next weeks some tools to run SfM reconstruction with SuperPoint and SuperGlue, but it is not ready yet.

LimHyungTae commented 4 years ago

I appreciate your quick reply, and I found the fact that the number of 3-D pointclouds increases when we set appropriate NMS value of superpoint! In detail, the number of points become almost 80,000 -> 235,562. So it might indicate that redundant keypoints gathered by superpoint (originally, NMS value was set to 0) hinder the matching of images. Even though it is improved, yet the # of points are still few in number compared to pre-constructed SfM model (680,000) that you provide. Do you have any insight to resolve this issue? Thank you and have a nice weekend :)

LimHyungTae commented 4 years ago

model_img => SfM that you uploaded 스크린샷, 2020-06-19 21-13-47 => Ours :(

sarlinpe commented 4 years ago

What version of Tensorflow and PyTorch are you using? Can you visualize keypoints in the npz files?

LimHyungTae commented 4 years ago

I use TF-GPU 1.15 and 1.3 pytorch. 118 And visualized points seem to be reasonable, as far as I understood (actually, some points are redundant). Is there any mistake?? We just revise NMS option 0 to 4. I upload visualized imgs and npz files also.

LimHyungTae commented 4 years ago

1.zip 2.zip

LimHyungTae commented 4 years ago

3.zip 4.zip

sarlinpe commented 4 years ago

The detections look reasonable - the NMS should instead be set to 4. Checking the matching would help, there is debug flag in match_features_with_db_prior.py- it will plot the matches.

LimHyungTae commented 4 years ago

When we checked the results of matchings, the matchings' are fine. We suspect several factors: i) in python3 update_db_with_nvm_intrinsics.py, should we revise focal length since the images are resized to 960? ii) in colmap feature_importer, should we resize the images to set correct relationships btw keypoints and images (yet I believe images are only employed when colorization)?

LimHyungTae commented 4 years ago

0 50 1050 3250 3850

LimHyungTae commented 4 years ago

Oops, I found why the result is off! How silly I am....I skipped colmap-helpers/export_for_sfm.py procedure...lol. Sorry for bothering you and I really appreciate your kind help! Thank you. :)

sarlinpe commented 4 years ago

Glad you found the issue, I hope the next steps will go smoothly for you! :)