Error when using Roma

[antithing]

Hi, and thanks for this code! I have installed in conda and am running the command:

python main.py --dir assets/test --pipeline roma

Features are extracted, but the matcher fails with:

2024-10-04 10:33:28 | [INFO    ] Features extracted!
2024-10-04 10:33:28 | [INFO    ] Matching features with roma...
2024-10-04 10:33:28 | [INFO    ] roma configuration:
{'name': 'roma', 'pretrained': 'outdoor'}
2024-10-04 10:33:28 | [INFO    ] Matching features...
2024-10-04 10:33:28 | [INFO    ]
  0%|                                                                                            | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Users\B\deep-image-matching\main.py", line 49, in <module>
    match_path = img_matching.match_pairs(feature_path)
  File "C:\Users\B\deep-image-matching\src\deep_image_matching\image_matching.py", line 427, in match_pairs
    self._matcher.match(
  File "C:\Users\B\deep-image-matching\src\deep_image_matching\matchers\roma.py", line 89, in match
    matches = self._match_pairs(self._feature_path, img0, img1)
  File "F:\Conda\envs\deep-image-matching\lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\B\deep-image-matching\src\deep_image_matching\matchers\roma.py", line 165, in _match_pairs
    warp, certainty = self.matcher.match(
  File "F:\Conda\envs\deep-image-matching\lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\B\deep-image-matching\src\deep_image_matching\thirdparty\RoMa\roma\models\matcher.py", line 709, in match
    im_A, im_B = test_transform((im_A, im_B))
  File "C:\Users\B\deep-image-matching\src\deep_image_matching\thirdparty\RoMa\roma\utils\utils.py", line 292, in __call__
    im_tuple = t(im_tuple)
  File "C:\Users\B\deep-image-matching\src\deep_image_matching\thirdparty\RoMa\roma\utils\utils.py", line 209, in __call__
    return [self.to_tensor(im) for im in im_tuple]
  File "C:\Users\B\deep-image-matching\src\deep_image_matching\thirdparty\RoMa\roma\utils\utils.py", line 209, in <listcomp>
    return [self.to_tensor(im) for im in im_tuple]
  File "C:\Users\B\deep-image-matching\src\deep_image_matching\thirdparty\RoMa\roma\utils\utils.py", line 194, in __call__
    im = np.array(im, dtype=np.float32).transpose((2, 0, 1))
ValueError: axes don't match array

I am attaching the config here as well. What can I do to resolve this? Thanks!

config.json

[lcmrl]

Hi, thanks for reporting, are you on dev branch? Could you just try to run this basic example to see if you have the same issue: >python ./main.py -d .\assets\example_cyprus -p roma --skip_reconstruction --force

[antithing]

I installed via pip, inside conda. Will check with that command and report back. Thanks!

[antithing]

Hi, I think I have the issue... using the cypruss dataset, things look okay, using a custom dataset where the images are of different sizes, I get the error.

Could the different sizes be the problem here? How can I get around it?

[lcmrl]

Hi, what is the size of the images?

[antithing]

it is a 3 image dataset.

2 x 1280 x 800 1 x 1920 x 1080

[antithing]

more testing... actually even if i remove the 1920x1080 image, i see the same error. When using Loftr, it runs perfectly. This is specific to a roma pipeline

[lcmrl]

Are your images grayscale or RGB?

[antithing]

greyscale

[lcmrl]

Could you try with any RGB images that you have? Maybe this is the issue. In that case we could convert grayscale to RGB and this should solve the problem