problem in 3D recon on ETH. Points shifted from reference images

[GabbySuwichaya]

Hi ! Thank you for the great work and sharing! ASLfeat is a very competitive method and is currently one of the best in MMA.

So, I have been interested in the ASLfeat's performance on 3D reconstruction as it seems that ASLfeat is doing very well in providing high registered images and sparse points.

So, I run a small test using the output from your package and integrate it into ETH Benchmark evaluation on Herzjesu.

• The problem is that I have got the following results. The sparse points projected on images are shifted from where it supposed to be (as shown in the image number 8 on the right).

Could you advice if I should set anything in addition ?

Here is the results of ASLfeat.

Also, just to give a reference. Here is the results of SIFT. The other methods seem to do ok too.

Also, I attached my setting in the config here. If it happened that I did not set something right, please let me know:.

data_name: 'eth'
data_split: ['Herzjesu', 'Fountain', 'South-Building']  #['Gendarmenmarkt', 'Madrid_Metropolis', 'Tower_of_London']
data_root: '/mnt/HDD4TB1/local-feature-evaluation/datasets'
dump_root: 
truncate: [0, null]
model_path: 'pretrained/aslfeat/model.ckpt-380000' # 'pretrained/aslfeat/model.ckpt-60000'  # 
overwrite: true
net:
  max_dim: 2048 #1600 #
  config:
    kpt_n: 20000 # 20000
    kpt_refinement: true
    deform_desc: 1
    score_thld: 0.5
    edge_thld: 10
    multi_scale: true
    multi_level: true
    nms_size: 3
    eof_mask: 5
    need_norm: true
    use_peakiness: true
post_format:
  suffix: '_aslfeat20K2048_380'

[GabbySuwichaya]

Just to update. I think I found a big bug in your code... In L 59, feat_model, you have made an extract scaling which leads to the wrong scaling....
So, the scalling np.array([W / data_size[1], H / data_size[0]], dtype=np.float32) should be removed.

So, instead of this ..... kpts = kpts[idxs] *** np.array([W / data_size[1], H / data_size[0]], dtype=np.float32)**

It should be this .... kpts = kpts[idxs]

[vdvchen]

Hi,

Thank you very much for this observation! We checked the code and find this is indeed a scaling bug. We have fixed it in the latest commit. This bug causes issues when input size exceeds config['max_dim'], which is set as 2048 by default. We will investigate the bug's influence on our previous results.

The shifting issue in your case mighted be caused by this scaling bug. Could you please try our latest code and tell us if the probelm is solved?

[GabbySuwichaya]

Thanks for letting me know. I have seen the commit id 4ca6cf4.... You have removed that line and that fixed the issue on 3D recon. ....But that is as far as I know.

[DHNicoles]

@GabbySuwichaya Hi, When you do 3D reconstruction, what kind of matching method did you using on ASLfeat?

[GabbySuwichaya]

@DHNicoles I used the NN Matching, just like those in https://github.com/ahojnnes/local-feature-evaluation.

[DHNicoles]

Thanks! I have used SuperPoint+SuperGlue for 3D reconstruction, and visualized with colmap, it works well. So I'm a little interested in the result of ASLFeat+SuperGlue, but no relevant discussion was found.

[GabbySuwichaya]

@DHNicoles I am not sure about the size of ASLFeat's descriptor... So far what I know is that SuperGlue was trained with SuperPoints, and the pre-trained model is based on the descriptor size of 256.

Moreover, if you read SuperGlue's paper carefully, you'll find that the SuperGlue has been tested with SuperPoint as well as SIFT. In some way, you may need to train SuperGlue + ASLFeat features.....by yourself..... as far as I can remember. The author of SuperGlue does not release the training script. [But if you find one, I would appreciate if you can share the code.]

However, I think the match of SuperGlue + ASLFeat should work well, because ASLFeat has very high repeatability, just like SuperPoint [This is an underlying condition of SuperGlue].

[x1597275]

Hi ! Thank you for the great work and sharing! ASLfeat is a very competitive method and is currently one of the best in MMA.

So, I have been interested in the ASLfeat's performance on 3D reconstruction as it seems that ASLfeat is doing very well in providing high registered images and sparse points.

So, I run a small test using the output from your package and integrate it into ETH Benchmark evaluation on Herzjesu.

• The problem is that I have got the following results. The sparse points projected on images are shifted from where it supposed to be (as shown in the image number 8 on the right).

Could you advice if I should set anything in addition ?

Here is the results of ASLfeat.

Also, just to give a reference. Here is the results of SIFT. The other methods seem to do ok too.

Also, I attached my setting in the config here. If it happened that I did not set something right, please let me know:.

data_name: 'eth'
data_split: ['Herzjesu', 'Fountain', 'South-Building']  #['Gendarmenmarkt', 'Madrid_Metropolis', 'Tower_of_London']
data_root: '/mnt/HDD4TB1/local-feature-evaluation/datasets'
dump_root: 
truncate: [0, null]
model_path: 'pretrained/aslfeat/model.ckpt-380000' # 'pretrained/aslfeat/model.ckpt-60000'  # 
overwrite: true
net:
  max_dim: 2048 #1600 #
  config:
    kpt_n: 20000 # 20000
    kpt_refinement: true
    deform_desc: 1
    score_thld: 0.5
    edge_thld: 10
    multi_scale: true
    multi_level: true
    nms_size: 3
    eof_mask: 5
    need_norm: true
    use_peakiness: true
post_format:
  suffix: '_aslfeat20K2048_380'

hello,I'm a beginner ,can you tell me how to use the aslfeat in colmap ? I can use colmap but don't konw how to use aslfeat in colmap together.

[GabbySuwichaya]

Hi, I follow the standard way ....https://github.com/ahojnnes/local-feature-evaluation.
So, please check https://github.com/ahojnnes/local-feature-evaluation/blob/master/INSTRUCTIONS.md

[x1597275]

Hi, I follow the standard way ....https://github.com/ahojnnes/local-feature-evaluation. So, please check https://github.com/ahojnnes/local-feature-evaluation/blob/master/INSTRUCTIONS.md

than you very much!

[x1597275]

Hi, I follow the standard way ....https://github.com/ahojnnes/local-feature-evaluation. So, please check https://github.com/ahojnnes/local-feature-evaluation/blob/master/INSTRUCTIONS.md

Hi bother you again,I follow the instructions to build the colmap successful,and I use aslfeat compute the keypoints and descriptor like this: but when I run the vocab_tree_builder_float,it reports this:

I debug the code found that when the code read database to get the images,there are no images in the database.db,can you tell me what you did when run the 'vocab_tree_builder_float'?

[GabbySuwichaya]

Hi! I am not quite sure about this ....

I think it is best to post this question to the original repo: https://github.com/ahojnnes/local-feature-evaluation ... It is more related to how to set up descriptor size and build COLMAP ( I guess) ....

Also, maybe you should check this one as well. I also use the following repo for using COLMAP with Custom Features: https://github.com/tsattler/visuallocalizationbenchmark/tree/master/local_feature_evaluation#using-colmap-with-custom-features https://github.com/tsattler/visuallocalizationbenchmark/tree/master/local_feature_evaluation

Or, maybe you should tag the author of the ASLFeat repo, as well. I am not a contributor; I am just a loyal fan .... just like you :) .

[x1597275]

Hi! I am not quite sure about this ....

I think it is best to post this question to the original repo: https://github.com/ahojnnes/local-feature-evaluation ... It is more related to how to set up descriptor size and build COLMAP ( I guess) ....

Also, maybe you should check this one as well. I also use the following repo for using COLMAP with Custom Features: https://github.com/tsattler/visuallocalizationbenchmark/tree/master/local_feature_evaluation#using-colmap-with-custom-features https://github.com/tsattler/visuallocalizationbenchmark/tree/master/local_feature_evaluation

Or, maybe you should tag the author of the ASLFeat repo, as well. I am not a contributor; I am just a loyal fan .... just like you :) .

OK，I'll try,thank you!