search

introlab / rtabmap

RTAB-Map library and standalone application
https://introlab.github.io/rtabmap
Other
2.61k stars 761 forks source link

SuperGlue #599

Open lobantseff opened 3 years ago

lobantseff commented 3 years ago

Hello, Mathieu!

I'm trying to run SuperGlue matching using the parameter Vis/CorNNType: 6 and encounter the issue as follows:

[ERROR] (2020-09-16 09:45:32.632) PyMatcher.cpp:109::PyMatcher() Module "rtabmap_superglue" could not be imported! (File="/home/artem/packages/SuperGluePretrainedNetwork/rtabmap_superglue.py")

I've changed the rtabmap_superglue.py shebang line to the python from the conda env with PyTorch installed, and added sys.version print, but got the same issue as above:

('_PATH', ['/home/artem/arrival-vslam/catkin_ws/devel/lib/python2.7/dist-packages', '/home/artem/bg-robot/catkin_ws/devel/lib/python2.7/dist-packages', '/opt/ros/melodic/lib/python2.7/dist-packages', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/artem/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/wx-3.0-gtk3', '/home/artem/packages/SuperGluePretrainedNetwork'])
('_VERSION', '2.7.17 (default, Jul 20 2020, 15:37:01) \n[GCC 7.5.0]')
NUMPY
[ERROR] (2020-09-16 09:45:32.632) PyMatcher.cpp:109::PyMatcher() Module "rtabmap_superglue" could not be imported! (File="/home/artem/packages/SuperGluePretrainedNetwork/rtabmap_superglue.py")

I suppose, that the problem in the Python version. Because of Python 2.7 it cannot import PyTorch.

  1. How can I use the python version from the environment with torch, or even python3 version in rtabmap SuperGlue matching?
  2. Do I need to change Vis/FeatureTypeto 10 (SuperPoint) during the Superglue matching to use SuperPoint+SuperGlue bundle, or Superglue uses SP by default (because of the pre-trained SP weights in the repo)

Thanks for your time! Best wishes, Artem

matlabbe commented 3 years ago

For SuperGlue, it is quite long and painful to make it work on system that has Python2 by default (because some dependencies of rtabmap were built with python2 and this is causing crashes when we force rtabmap linking on Python3 at the same time). Can you try on Ubuntu 20.04?

For the other questions, only SuperPoint will work with SuperGlue as SuperGlue's models (indoor and outdoor) that they provide are based only on SuperPoint. They didn't publicly released their model for SIFT for example. If you managed to train another SuperGlue model on another descriptor type, you would have to change the script here to hardcode some other value: https://github.com/introlab/rtabmap/blob/3d33370e4c5530c964ba810ca4639dbaa520f342/corelib/src/pymatcher/rtabmap_superglue.py#L32 or remove this assert to pass anything else to PyMatcher/Model rtabmap's parameter: https://github.com/introlab/rtabmap/blob/3d33370e4c5530c964ba810ca4639dbaa520f342/corelib/src/pymatcher/rtabmap_superglue.py#L29

cheers, Mathieu

lobantseff commented 3 years ago

Ok, I’ll try in docker and will tell if I’ll gain success.

So, did I correctly understand that I don’t need to change Vis/FeatureType: to 10 (Superpoint) if I use Vis/CorNNType: 6 (superglue)?

Best regards, Artem

On 16 Sep 2020, at 18:03, matlabbe notifications@github.com wrote:

For SuperGlue, it is quite long and painful to make it work on system that has Python2 by default (because some dependencies of rtabmap were built with python2 and this is causing crashes when we force rtabmap linking on Python3 at the same time). Can you try on Ubuntu 20.04?

For the other questions, only SuperPoint will work SuperGlue as SuperGlue's model that they provide is based only on SuperPoint. They didn't publicly released their model for SIFT for example. If you managed to train another SuperGlue model on another descriptor type, you would have to change the script here to hardcode some other value: https://github.com/introlab/rtabmap/blob/3d33370e4c5530c964ba810ca4639dbaa520f342/corelib/src/pymatcher/rtabmap_superglue.py#L32 https://github.com/introlab/rtabmap/blob/3d33370e4c5530c964ba810ca4639dbaa520f342/corelib/src/pymatcher/rtabmap_superglue.py#L32 or remove this assert to pass anything else: https://github.com/introlab/rtabmap/blob/3d33370e4c5530c964ba810ca4639dbaa520f342/corelib/src/pymatcher/rtabmap_superglue.py#L29 https://github.com/introlab/rtabmap/blob/3d33370e4c5530c964ba810ca4639dbaa520f342/corelib/src/pymatcher/rtabmap_superglue.py#L29 cheers, Mathieu

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/introlab/rtabmap/issues/599#issuecomment-693466277, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFMBN3E3ZGSCM7HVUK3SXYLSGDHS3ANCNFSM4ROK52QA.

matlabbe commented 3 years ago

Kp/DetectorStrategy, Vis/FeatureType should be set to 11 (SuperPoint) if Vis/CorNNType is set to 6 (SuperGlue).

If you want odometry computed with default features as before (may be faster to compute than SuperPoint) but for loop closure detection using SuperGlue with SuperPoint, keep Vis/FeatureType to its default. Odometry will swicth to default matching if it detects non supported descriptors ofr SuperGlue.

fbliman commented 10 months ago

I am having this same problem


 [ERROR] (2023-08-09 17:01:16.040) PyMatcher.cpp:61::PyMatcher() Module "rtabmap_superglue" could not be imported! (File="/home/fede/rtabmap/archive/2022-IlluminationInvariant/scripts/SuperGluePretrainedNetwork/rtabmap_superglue.py")
[ERROR] (2023-08-09 17:01:16.040) PyMatcher.cpp:62::PyMatcher() Traceback (most recent call last):

  File "/home/fede/rtabmap/archive/2022-IlluminationInvariant/scripts/SuperGluePretrainedNetwork/rtabmap_superglue.py", line 10, in <module>
    import torch

  File "/home/fede/.local/lib/python3.8/site-packages/torch/__init__.py", line 229, in <module>
    from torch._C import *  # noqa: F403

ImportError: /home/fede/.local/lib/python3.8/site-packages/torch/lib/libtorch_python.so: undefined symbol: _ZTIN4c10d4WorkE

[ERROR] (2023-08-09 17:01:16.052) PyMatcher.cpp:61::PyMatcher() Module "rtabmap_superglue" could not be imported! (File="/home/fede/rtabmap/archive/2022-IlluminationInvariant/scripts/SuperGluePretrainedNetwork/rtabmap_superglue.py")
[ERROR] (2023-08-09 17:01:16.052) PyMatcher.cpp:62::PyMatcher() Traceback (most recent call last):

  File "/home/fede/rtabmap/archive/2022-IlluminationInvariant/scripts/SuperGluePretrainedNetwork/rtabmap_superglue.py", line 10, in <module>
    import torch

  File "/home/fede/.local/lib/python3.8/site-packages/torch/__init__.py", line 229, in <module>
    from torch._C import *  # noqa: F403

ImportError: /home/fede/.local/lib/python3.8/site-packages/torch/lib/libtorch_python.so: undefined symbol: _ZTIN4c10d4WorkE

[ERROR] (2023-08-09 17:01:16.062) PyMatcher.cpp:61::PyMatcher() Module "rtabmap_superglue" could not be imported! (File="/home/fede/rtabmap/archive/2022-IlluminationInvariant/scripts/SuperGluePretrainedNetwork/rtabmap_superglue.py")
[ERROR] (2023-08-09 17:01:16.062) PyMatcher.cpp:62::PyMatcher() Traceback (most recent call last):

  File "/home/fede/rtabmap/archive/2022-IlluminationInvariant/scripts/SuperGluePretrainedNetwork/rtabmap_superglue.py", line 10, in <module>
    import torch

  File "/home/fede/.local/lib/python3.8/site-packages/torch/__init__.py", line 229, in <module>
    from torch._C import *  # noqa: F403

ImportError: /home/fede/.local/lib/python3.8/site-packages/torch/lib/libtorch_python.so: undefined symbol: _ZTIN4c10d4WorkE

[ERROR] (2023-08-09 17:01:16.269) PyMatcher.cpp:61::PyMatcher() Module "rtabmap_superglue" could not be imported! (File="/home/fede/rtabmap/archive/2022-IlluminationInvariant/scripts/SuperGluePretrainedNetwork/rtabmap_superglue.py")
[ERROR] (2023-08-09 17:01:16.269) PyMatcher.cpp:62::PyMatcher() Traceback (most recent call last):

  File "/home/fede/rtabmap/archive/2022-IlluminationInvariant/scripts/SuperGluePretrainedNetwork/rtabmap_superglue.py", line 10, in <module>
    import torch

  File "/home/fede/.local/lib/python3.8/site-packages/torch/__init__.py", line 229, in <module>
    from torch._C import *  # noqa: F403

ImportError: /home/fede/.local/lib/python3.8/site-packages/torch/lib/libtorch_python.so: undefined symbol: _ZTIN4c10d4WorkE

I am in ubuntu 20.04 with python 3.8

Thanks!!!

matlabbe commented 10 months ago

Looks like pytorch is not found. for the libtorch_python.so: undefined symbol, cannot really say. I have always a hard time to correctly install pytorch to mkae it work at the same time for cpp and python, so I am using docker now: https://github.com/introlab/rtabmap/blob/master/docker/frontiers2022/Dockerfile

fbliman commented 10 months ago

Thanks!! ok, I will try docker then.

For rtabmap-ros with superpotin and superglue I guess I should use https://github.com/introlab/rtabmap_ros/tree/master/docker/noetic/superpoint

matlabbe commented 10 months ago

@fbliman yes that docker image.