Run example.py but get empty matches

[541979210]

hello, I download the code and run example.py but get empty matches, Is anyone having the same problem?

[cavalli1234]

Hello, it is possible that AdaLAM with default parameters does not leave any match through for particularly challenging image pairs. This is especially the case when there are few keypoints available already in input (e.g. low resolution images, poorly textured images), as they may not be enough to form significant enough statistics altogether.

To make AdaLAM to work on an image set where few keypoints can be extracted it may be convenient to tweak the default parameters (they are tuned for image pairs with 2k-8k keypoints per image)

Let me know if you need any help!

[541979210]

Thanks for your reply. Actually in my case,I only test the example image pair(im1.jpg,im2.jpg),and the extrated features seem fine(8k kps),so I dont know whats wrong with the matching method. Is there anything need to know?

[cavalli1234]

This is unexpected, as the example pair should work fine without any tuning.

I would like to reproduce the issue, can you provide a description of your environment? (pip freeze/conda list)

[541979210]

Here is my package environment. Everything looks fine before line 119 in core.py which I got im1loc and im2loc shape 3880, rdims shape 42.But after function ransac,the inlier_idx got shape 0. absl-py==0.7.1 AdaLAM==0.1 astor==0.8.0 attrs==19.3.0 backcall==0.1.0 bleach==1.5.0 boto3==1.12.4 botocore==1.15.4 cachetools==4.2.0 certifi==2019.6.16 chardet==3.0.4 cycler==0.10.0 decorator==4.4.0 defusedxml==0.6.0 docutils==0.15.2 easydict==1.9 entrypoints==0.3 enum34==1.1.10 faiss==1.6.0 freetype-py==2.1.0.post1 gast==0.2.2 get==2019.4.13 google-auth==1.24.0 grpcio==1.21.1 h5py==2.9.0 html5lib==0.9999999 idna==2.9 imageio==2.5.0 importlib-metadata==0.23 ipykernel==5.1.3 ipython==7.9.0 ipython-genutils==0.2.0 ipywidgets==7.5.1 jedi==0.15.1 Jinja2==2.10.3 jmespath==0.9.4 jsonschema==3.1.1 jupyter-client==5.3.4 jupyter-core==4.6.1 Keras==2.3.1 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.0 kiwisolver==1.1.0 lxml==4.5.0 Markdown==3.1.1 MarkupSafe==1.1.1 matplotlib==3.1.3 mistune==0.8.4 mkl-fft==1.0.12 mkl-random==1.0.2 mkl-service==2.0.2 more-itertools==7.2.0 moxing-pytorch==1.13.0 nbconvert==5.6.1 nbformat==4.4.0 networkx==2.3 neural-renderer-pytorch==1.1.3 notebook==6.0.1 numpy==1.19.0 nvidia-ml-py==10.418.84 onnx==1.7.0 onnx-tf==1.5.0 open3d==0.9.0.0 opencv-contrib-python==3.4.2.16 opencv-python==4.1.0.25 ordereddict==1.1 pandas==0.24.2 pandocfilters==1.4.2 parse==1.12.0 parso==0.5.1 pathlib2==2.3.5 pexpect==4.7.0 pickleshare==0.7.5 Pillow==6.0.0 post==2019.4.13 progress==1.5 prometheus-client==0.7.1 prompt-toolkit==2.0.10 protobuf==3.9.1 psutil==5.7.2 ptyprocess==0.6.0 public==2019.4.13 pyasn1==0.4.8 pyasn1-modules==0.2.8 pydegensac==0.1.2 pyglet==1.5.5 Pygments==2.4.2 PyOpenGL==3.1.0 pyparsing==2.4.2 pyrender==0.1.43 pyrsistent==0.15.5 python-dateutil==2.8.0 pytz==2019.1 PyWavelets==1.0.3 PyYAML==5.3 pyzmq==18.1.0 query-string==2019.4.13 requests==2.23.0 rsa==4.6 s3transfer==0.3.3 scikit-image==0.15.0 scipy==1.5.1 Send2Trash==1.5.0 sharedmem==0.3.7 six==1.12.0 tabulate==0.8.7 tensorboard==1.8.0 tensorboard-plugin-wit==1.7.0 tensorboardX==2.0 tensorflow-gpu==1.8.0 termcolor==1.1.0 terminado==0.8.2 testpath==0.4.2 torch==1.2.0 torch-dimcheck==0.0.1 torch-localize==0.1.0 torchfile==0.1.0 torchvision==0.2.1 tornado==6.0.3 tqdm==4.32.2 traitlets==4.3.3 trimesh==3.6.43 typing-extensions==3.7.4.2 urllib3==1.25.8 wcwidth==0.1.7 Werkzeug==0.15.4 wget==3.2 widgetsnbextension==3.5.1 zipp==0.6.0

[cavalli1234]

I tried reproducing the issue, and it seems that with pytorch==1.2.0 as you are using the existing code raised exception on an implicit cast which was not supported for that version. It was an easy fix, which is now integrated into adalam for better compatibility (Thanks!).

Still, I do not yet understand how it is possible that it did not produce matches for you instead of raising the error. Did you fix the issue yourself? After the fix, I could get the expected matches for the example pair as with more recent torch versions.

Could you please try to update adalam and try again?

[541979210]

After update adalam , I could get the expected matches for the example image pair.But in my matching pair, it still shows no matches. And I find that for the same image input,like two im1.jpg input,theoretically algorithm should get many matches,becase they are the same image,but actually it got no match. In my test, my query kps number is 3274, match kps number is 672,and they both have 512-dimentional descriptors(extracted by D2NET).

[541979210]

I change the matching pairs.It shows that in some cases,the algorithm works fine. Still I dont know why it fails in particular image.Anyway,thanks for your help

[cavalli1234]

Regarding the test of matching an image with itself, for adalam it represents an edge case as every single correspondence matches perfectly, so no one is filtered out and the memory footprint of the whole operation becomes much higher than a "regular" matching pair. This is why you should expect a memory error to be raised if you try to match a high resolution image with itself in the current version of adalam which does not handle this edge case explicitly. In particular with the example images the memory requirements are higher than any reasonable GPU memory.

Given that the update made it work for you and that in both of these cases you don't see matches in output where you should see a specific error (I refer to the invalid casting error due to torch 1.2.0 before, and the expected memory error in this case) I suspect there is something wrong with exception handling in your environment. These errors are not explicitly silenced by adalam, and they are expected to be raised up instead of producing empty matches. Could you check what happens in your case when some exception is raised from the adalam functions?

One more note if you plan to use adalam in conjunction with D2NET: the default parameters are tuned for SIFT, which usually provides quite well localized keypoints with orientation and scale information. This means that with learned detectors generally the input to adalam is much more noisy (especially if orientation and scale are missing as pre-filters) and some tuning of the default parameters may improve performance.

[cavalli1234]

Closing for now, feel free to re-open!