score0 missing?

[dberga]

I am using several repos related to nerfs (with sdfstudio, nerfstudio...) and they run hloc / colmap for the preprocessing the sfm data.

In the matcher part I get an error with regards not finding the key score0 in data. I am using disk features. The error appears in https://github.com/cvg/Hierarchical-Localization/blob/master/hloc/utils/base_model.py#L24

By simply running ns-process-data images --data data/heritage_data_patrick_sdf/hotel_international --output-dir data/heritage_data_patrick_sdf/hotel_international --camera_type perspective --matching-method exhaustive --feature-type disk --matcher-type any

I always get:

[2024/04/09 17:05:33 hloc INFO] Extracting local features with configuration:
{'model': {'max_keypoints': 5000, 'name': 'disk'},
 'output': 'feats-disk',
 'preprocessing': {'grayscale': False, 'resize_max': 1600}}
[2024/04/09 17:05:33 hloc INFO] Skipping the extraction.
[2024/04/09 17:05:33 hloc INFO] Found 153 pairs.
[2024/04/09 17:05:33 hloc INFO] Matching local features with configuration:
{'model': {'name': 'superglue',
           'sinkhorn_iterations': 50,
           'weights': 'outdoor'},
 'output': 'matches-superglue'}
  0%|                                                                                                                                                                              | 0/153 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/Imatge/projects/ai4heritage/Hierarchical-Localization/hloc/utils/base_model.py", line 25, in forward
    assert key in data, "Missing key {} in data".format(key)
AssertionError: Missing key scores0 in data

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/Imatge/miniconda3/envs/sdfstudio/bin/ns-process-data", line 8, in <module>
    sys.exit(entrypoint())
  File "/home/Imatge/projects/ai4heritage/sdfstudio/scripts/process_data.py", line 797, in entrypoint
    tyro.cli(Commands).main()
  File "/home/Imatge/projects/ai4heritage/sdfstudio/scripts/process_data.py", line 122, in main
    hloc_utils.run_hloc(
  File "/home/Imatge/projects/ai4heritage/sdfstudio/nerfstudio/process_data/hloc_utils.py", line 96, in run_hloc
    match_features.main(matcher_conf, sfm_pairs, features=features, matches=matches)
  File "/home/Imatge/projects/ai4heritage/Hierarchical-Localization/hloc/match_features.py", line 174, in main
    match_from_paths(conf, pairs, matches, features_q, features_ref, overwrite)
  File "/home/Imatge/miniconda3/envs/sdfstudio/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/Imatge/projects/ai4heritage/Hierarchical-Localization/hloc/match_features.py", line 244, in match_from_paths
    pred = model(data)
  File "/home/Imatge/miniconda3/envs/sdfstudio/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/Imatge/projects/ai4heritage/Hierarchical-Localization/hloc/utils/base_model.py", line 32, in forward
    assert key in data, "Missing key {} in data".format(key)
AssertionError: Missing key scores0 in data

Tried with distinct --matcher-type among any,NN,superglue,superglue-fast,NN-superpoint,NN-ratio,NN-mutual,adalam,disk+lightglue,superpoint+lightglue but still having the same error.

Here are the values I printed when debugging:

key = scores0

data.keys() = dict_keys(['descriptors0', 'image_size0', 'keypoint_scores0', 'keypoints0', 'image0', 'descriptors1', 'image_size1', 'keypoint_scores1', 'keypoints1', 'image1'])

self.required_inputs = ['image0', 'keypoints0', 'scores0', 'descriptors0', 'image1', 'keypoints1', 'scores1', 'descriptors1']

[sarlinpe]

disk features are compatible only with matchers NN-mutual and disk+lightglue. We don't have a SuperGlue model trained for DISK features. I will add a check to flag this error earlier.

[dberga]

disk features are compatible only with matchers NN-mutual and disk+lightglue. We don't have a SuperGlue model trained for DISK features. I will add a check to flag this error earlier.

Tried also with disk+lightglue but the code crashes by having an almost empty dict of "image_ids" in https://github.com/cvg/Hierarchical-Localization/blob/master/hloc/triangulation.py#L96 not being able to run the import_matches.

[dberga]

Just made a pull request that fix the error of empty image_ids https://github.com/nerfstudio-project/nerfstudio/pull/3081