search

3DOM-FBK / deep-image-matching

Multiview matching with deep-learning and hand-crafted local features for COLMAP and other SfM software. Supports high-resolution formats and images with rotations. Both CLI and GUI are supported.
https://3dom-fbk.github.io/deep-image-matching/
BSD 3-Clause "New" or "Revised" License
338 stars 40 forks source link

matches with geometric verification? #57

Closed pengxutao closed 6 months ago

pengxutao commented 6 months ago

hello, in clomap, after the feature matching , it will also undergo geometric verification to remove the outlier . I want to know whether it will also undergo geometric verification in deep-image-matching, thank you.

lcmrl commented 6 months ago

Yes sure, you have different approaches for geometric verification, and default options uses pydegensac. If you are using DIM as library (see example here https://github.com/3DOM-FBK/deep-image-matching/blob/master/notebooks/sfm_pipeline.ipynb), you can change geometric verification parameters:

config.general["gv_threshold"] = 4
config.general["gv_confidence"] = 0.999
config.general["geom_verification"] = GeometricVerification.PYDEGENSAC

There is not only pydegensac (see https://github.com/3DOM-FBK/deep-image-matching/blob/master/src/deep_image_matching/utils/geometric_verification.py)

lcmrl commented 6 months ago

If instead you are launching by CLI, you can pass:

python ./main.py --dir /path/to/working/dir --pipeline superpoint+lightglue --config superpoint+lightglue.yaml

You have a config example in config folder where you can pass your options, for example:

# User configuration file

general:
  tile_size: (2400, 2000)
  geom_verification: pydegensac
  min_inliers_per_pair: 10
  min_inlier_ratio_per_pair: 0.25

extractor:
  name: "superpoint"
  max_keypoints: 8000 # -1 no limits
  nms_radius: 4
  keypoint_threshold: 0.005
  remove_borders: 4
  fix_sampling: False

matcher:
  name: "lightglue"
  flash: True # enable FlashAttention if available
  mp: False # enable mixed precision
  depth_confidence: 0.95 # early stopping, disable with -1
  width_confidence: 0.99 # point pruning, disable with -1
  filter_threshold: 0.10 # match threshold

All the default options of DIM are visible at https://github.com/3DOM-FBK/deep-image-matching/blob/master/src/deep_image_matching/config.py

pengxutao commented 6 months ago

how can I ban geometric verification? beacuse I want to test the origin matching ability without geometric verification. I try to use parameter 'none' in config file:

general:
  geom_verification: none

but here is an error:

Traceback (most recent call last):
  File "/home/xuzhi/pengxutao/deep-image-matching/main.py", line 49, in <module>
    match_path = img_matching.match_pairs(feature_path)
  File "/home/xuzhi/pengxutao/deep-image-matching/src/deep_image_matching/image_matching.py", line 427, in match_pairs
    self._matcher.match(
  File "/home/xuzhi/pengxutao/deep-image-matching/src/deep_image_matching/matchers/matcher_base.py", line 329, in match
    _, inlMask = geometric_verification(
  File "/home/xuzhi/pengxutao/deep-image-matching/src/deep_image_matching/utils/geometric_verification.py", line 128, in geometric_verification
    met = opencv_methods_mapping[method.name]
KeyError: 'NONE'

can you help me?

lcmrl commented 6 months ago

Maybe is a bug, in the meantime please set GeometricVerification.NONE in https://github.com/3DOM-FBK/deep-image-matching/blob/master/src/deep_image_matching/config.py

pengxutao commented 6 months ago

I modify the file https://github.com/3DOM-FBK/deep-image-matching/blob/master/src/deep_image_matching/config.py and set "geom_verification": GeometricVerification.NONE . But the error still exists:

Traceback (most recent call last):
  File "/home/xuzhi/pengxutao/deep-image-matching/main.py", line 49, in <module>
    match_path = img_matching.match_pairs(feature_path)
  File "/home/xuzhi/pengxutao/deep-image-matching/src/deep_image_matching/image_matching.py", line 427, in match_pairs
    self._matcher.match(
  File "/home/xuzhi/pengxutao/deep-image-matching/src/deep_image_matching/matchers/matcher_base.py", line 329, in match
    _, inlMask = geometric_verification(
  File "/home/xuzhi/pengxutao/deep-image-matching/src/deep_image_matching/utils/geometric_verification.py", line 128, in geometric_verification
    met = opencv_methods_mapping[method.name]
KeyError: 'NONE'
lcmrl commented 6 months ago

Hi, the issue should be now solved on branch dev. You can use for instance a config yaml file like this

# User configuration file

general:
  tile_size: (2400, 2000)
  geom_verification: PYDEGENSAC # NONE, PYDEGENSAC, MAGSAC, RANSAC, LMEDS, RHO, USAC_DEFAULT, USAC_PARALLEL, USAC_FM_8PTS, USAC_FAST, USAC_ACCURATE, USAC_PROSAC, USAC_MAGSAC
  gv_threshold: 4
  min_inliers_per_pair: 10
  min_inlier_ratio_per_pair: 0.25

extractor:
  name: "superpoint"
  max_keypoints: 8000 # -1 no limits
  nms_radius: 4
  keypoint_threshold: 0.005
  remove_borders: 4
  fix_sampling: False

matcher:
  name: "lightglue"
  flash: True # enable FlashAttention if available
  mp: False # enable mixed precision
  depth_confidence: 0.95 # early stopping, disable with -1
  width_confidence: 0.99 # point pruning, disable with -1
  filter_threshold: 0.10 # match threshold

In the next release the issue will be solved also in branch master. Thanks for the feedback