search

ros2 / rclpy

rclpy (ROS Client Library for Python)
Apache License 2.0
268 stars 221 forks source link

_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so failed to be imported while being present on the system #1144

Open squizz617 opened 11 months ago

squizz617 commented 11 months ago

Bug report

Required Info:

Steps to reproduce issue

$ file /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=7a13e46bccf18deb10a3cfdb76b436aa3856fdda, stripped


* Source installation:

$ source ~/ros2_humble/install/setup.sh $ python3 -c "import rclpy" Traceback (most recent call last): File "", line 1, in File "/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/init.py", line 49, in from rclpy.signals import install_signal_handlers File "/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/signals.py", line 15, in from rclpy.exceptions import InvalidHandle File "/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/exceptions.py", line 15, in from rclpy.impl.implementation_singleton import rclpy_implementation as _rclpy File "/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/impl/implementation_singleton.py", line 32, in rclpy_implementation = import_c_library('._rclpy_pybind11', package) File "/home/seulbae/ros2_humble/build/rpyutils/rpyutils/import_c_library.py", line 39, in import_c_library return importlib.import_module(name, package=package) File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ImportError: librcl_action.so: cannot open shared object file: No such file or directory The C extension '/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so' failed to be imported while being present on the system. Please refer to 'https://docs.ros.org/en/{distro}/Guides/Installation-Troubleshooting.html#import-failing-even-with-library-present-on-the-system' for possible solutions

$ file /home/seulbae/ros2_humble/build/rclpy/test_rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so /home/seulbae/ros2_humble/build/rclpy/test_rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=521156ac1b623401f2b012c6dcece989048e6273, not stripped



#### Expected behavior
`rclpy` successfully imported.

#### Actual behavior
Import failure.

#### Additional information
I'm not sure what's causing this issue. A ROS 2 scripts on my machine that worked before suddenly stopped working, and it turned out to be a import failure. Also, the trobleshooting guide in the error message seems to be only for Windows.
Thank you.
mjcarroll commented 11 months ago

What is the output of python3 -VV ?

squizz617 commented 11 months ago 
$ python3 -VV
Python 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0]
mjcarroll commented 11 months ago

Ah, I didn't read closely enough, this is likely the culprit: 

ImportError: librcl_action.so: cannot open shared object file: No such file or directory

Do you have ros-humble-rcl-action installed? You should also have the library located at /opt/ros/humble/librcl_action.so from the binaries.

clalancette commented 11 months ago

Further, what does your environment look like?

$ printenv | grep -i ros
squizz617 commented 11 months ago

I have ros-humble-rcl-action installed.

$ apt list | grep humble-rcl-action                                                                                           

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

ros-humble-rcl-action-dbgsym/jammy 5.3.3-1jammy.20230623.045652 amd64
ros-humble-rcl-action/jammy,now 5.3.3-1jammy.20230623.045652 amd64 [installed,automatic]

Env when using the binary installation:

$ printenv | grep -i ros          
PATH=/opt/ros/humble/bin:/usr/local/texlive/2022/bin/x86_64-linux:/home/seulbae/.local/bin:/usr/local/texlive/2022/bin/x86_64-linux:/home/seulbae/.local/bin:/opt/gcc-arm-none-eabi-9-2020-q2-update/bin:/home/seulbae/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
AMENT_PREFIX_PATH=/opt/ros/humble
PYTHONPATH=/opt/ros/humble/lib/python3.10/site-packages:/opt/ros/humble/local/lib/python3.10/dist-packages
LD_LIBRARY_PATH=/opt/ros/humble/opt/rviz_ogre_vendor/lib:/opt/ros/humble/lib/x86_64-linux-gnu:/opt/ros/humble/lib
ROS_DISTRO=humble
ROS_LOCALHOST_ONLY=0
ROS_PYTHON_VERSION=3
ROS_VERSION=2
squizz617 commented 11 months ago 
$ file /opt/ros/humble/lib/librcl_action.so 
/opt/ros/humble/lib/librcl_action.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=e1d3f18a4b279ed1c5c6aca4dd4332e5fd3f06e5, stripped
mjcarroll commented 11 months ago

I'm not seeing anything obvious. Can you also run ldd /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so

squizz617 commented 11 months ago 
seulbae@xps13 ~   [23-07-27 13:37:31]
$ source /opt/ros/humble/setup.zsh                                                                                
seulbae@xps13 ~   [23-07-27 13:37:32]
$ ldd /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so 
    linux-vdso.so.1 (0x00007fff79b99000)
    librcl_action.so => /opt/ros/humble/lib/librcl_action.so (0x00007f7f3459f000)
    librcl_lifecycle.so => /opt/ros/humble/lib/librcl_lifecycle.so (0x00007f7f34595000)
    libpython3.10.so.1.0 => /lib/x86_64-linux-gnu/libpython3.10.so.1.0 (0x00007f7f33e00000)
    librcl.so => /opt/ros/humble/lib/librcl.so (0x00007f7f34526000)
    librcl_yaml_param_parser.so => /opt/ros/humble/lib/librcl_yaml_param_parser.so (0x00007f7f3451a000)
    librmw_implementation.so => /opt/ros/humble/lib/librmw_implementation.so (0x00007f7f3450b000)
    librcl_logging_interface.so => /opt/ros/humble/lib/librcl_logging_interface.so (0x00007f7f34506000)
    librmw.so => /opt/ros/humble/lib/librmw.so (0x00007f7f344fa000)
    liblifecycle_msgs__rosidl_typesupport_c.so => /opt/ros/humble/lib/liblifecycle_msgs__rosidl_typesupport_c.so (0x00007f7f344f3000)
    librcutils.so => /opt/ros/humble/lib/librcutils.so (0x00007f7f344db000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f7f33a00000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7f344b9000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7f33600000)
    libtracetools.so => /opt/ros/humble/lib/libtracetools.so (0x00007f7f344b4000)
    liblifecycle_msgs__rosidl_generator_c.so => /opt/ros/humble/lib/liblifecycle_msgs__rosidl_generator_c.so (0x00007f7f344a4000)
    librosidl_runtime_c.so => /opt/ros/humble/lib/librosidl_runtime_c.so (0x00007f7f34499000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f7f34468000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f7f3444a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7f33d19000)
    librcl_logging_spdlog.so => /opt/ros/humble/lib/librcl_logging_spdlog.so (0x00007f7f34443000)
    librcl_interfaces__rosidl_typesupport_c.so => /opt/ros/humble/lib/librcl_interfaces__rosidl_typesupport_c.so (0x00007f7f3443a000)
    librcl_interfaces__rosidl_generator_c.so => /opt/ros/humble/lib/librcl_interfaces__rosidl_generator_c.so (0x00007f7f3441e000)
    libyaml.so => /opt/ros/humble/lib/libyaml.so (0x00007f7f343fc000)
    libament_index_cpp.so => /opt/ros/humble/lib/libament_index_cpp.so (0x00007f7f343ef000)
    librcpputils.so => /opt/ros/humble/lib/librcpputils.so (0x00007f7f343e1000)
    librosidl_typesupport_c.so => /opt/ros/humble/lib/librosidl_typesupport_c.so (0x00007f7f343db000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f7f34696000)
    libspdlog.so.1 => /lib/x86_64-linux-gnu/libspdlog.so.1 (0x00007f7f33c9e000)
    libbuiltin_interfaces__rosidl_generator_c.so => /opt/ros/humble/lib/libbuiltin_interfaces__rosidl_generator_c.so (0x00007f7f33c99000)
    libfmt.so.8 => /lib/x86_64-linux-gnu/libfmt.so.8 (0x00007f7f33c76000)
squizz617 commented 11 months ago

The issue persists after upgrading ros-humble apt package to ros-humble-rcl-action/jammy,now 5.3.4-3jammy.20230721.204515 amd64 [installed,automatic]..

mjcarroll commented 11 months ago

I'm pretty stumped here. I suppose as a next step, you could look at each of the dependent shared libraries and make sure that they can correctly resolve their dependencies via ldd?

iuhilnehc-ynos commented 11 months ago

@squizz617

I'm interested in this issue because it's almost impossible as the ldd shows that the rcl_action.so is found.

Could you run the command strace python3 -c "import rclpy" > /tmp/failed-to-import-rclpy.txt 2>&1, and then upload the log /tmp/failed-to-import-rclpy.txt?

squizz617 commented 11 months ago

@iuhilnehc-ynos Here's the strace of python3 importing rclpy: strace_rclpy.txt.

iuhilnehc-ynos commented 10 months ago

According to access("/etc/suid-debug", F_OK), I believe that you changed the /usr/bin/python3.10 with '+s' which will erase the LD_LIBRARY_PATH.

Please revert it with sudo chmod -s /usr/bin/python3.10.

squizz617 commented 10 months ago

I believe that you changed the /usr/bin/python3.10 with '+s'

@iuhilnehc-ynos Thanks for the suggestion but I haven't set the setuid bit.. 

$ ls -alh /usr/bin/python3.10 
-rwxr-xr-x 1 root root 5.7M Jun 11 01:26 /usr/bin/python3.10
iuhilnehc-ynos commented 10 months ago

Wow, so there are other cases can make the __libc_enable_secure get true and then call unsetenv the LD_LIBRARY_PATH.

Besides the manually setting suid for the /usr/bin/python3.10, another case I can think of is the mount option.

@squizz617 Could you show the mount options for the device?

DRAGON-BOOSTERR commented 10 months ago

I also have the same problem, for me the command import rclpy was not working in pycharm and after some messing around I found that we need to add the interpreter paths in pycharm. after that I ended up this error and by following this beautiful thread, Im looking for a way forward. but when I run the code with just the python command python3 ros_test.py, the program runs.

PS: I was following a udemy tutorial on ROS2

chenyenru commented 10 months ago

@DRAGON-BOOSTERR May you elaborate on how you solved the problem? I'm having the same problem too 😭

DRAGON-BOOSTERR commented 10 months ago

@chenyenru yeah here is the stackover flow page that I raised the issue and answered myself https://stackoverflow.com/questions/76983995/python-package-is-installed-but-it-doesnt-get-detected-in-a-ide-ros2/77009205#77009205

chenyenru commented 10 months ago

Thank you! I was able to solve it by deactivating my anaconda environment (don't know why but that way my Python interpreter finds rclpy package).

DRAGON-BOOSTERR commented 10 months ago

@chenyenru @squizz617 Hey guys I think i found a short term fix for this issue! As said before I was having problems with IDEs not recognising the rclpy library of ROS2. As by the comment yesterday I mentioned how I solved the problem by adding the interpreter paths in Pycharm IDE. The Pycharm IDE tries to run the file inside the IDE or something like that and received the same error as the title of this thread. Today I tried the code with visual studio code, surprisingly VS code recognised that rclpy library and showed all the auto complete and other things of that library. Since VS code runs every file in terminal inside the IDE, so the code works without any issues. I think this might fix your problem with that error?

chenyenru commented 10 months ago

I was able to include extra paths in the settings.json file, where the interpreter would successfully autosuggest functions from the rclpy library.

However, when I tried to compile it on the IDE, it still showed that rclpy was not found. I was able to resolve this problem by not running it on conda environment and instead the base file while installing required packages with pip. Hope this result would help you too!

fferri commented 9 months ago

I got the exact issue reported by OP. Turned out I was overriding LD_LIBRARY_PATH for the process responsible for spawning python3. Appending/prepending paths to LD_LIBRARY_PATH -instead of overriding- fixed the issue for me.

asukiaaa commented 8 months ago

I could resolve this problem by removing all packages of using ros2 distribution and install again.

sudo apt remove ros-rolling-*
sudo apt install ros-rolling-desktop-full
2019lelexia commented 7 months ago

According to this, https://docs.ros.org/en/rolling/How-To-Guides/Installation-Troubleshooting.html#import-failing-without-library-present-on-the-system I think we cannot use different version of Python because the Python of ROS2 build different binary file of C extension from the Python of another

yerkgb commented 7 months ago

I had a same problem. These two lines are helped me out: $sudo apt install --reinstall ros-humble-rclpy $source /opt/ros/humble/setup.bash Try your luck!

Also, on your .bashrc file try to source ros directory at the end after all lines.

sungchan1 commented 5 months ago

I finally managed to solve the problem and wrote a blog post about it in Korean.

https://du-sungchan-24k.tistory.com/37

In summary, you need to directly inject this into your pycharm.sh script.

. /opt/ros/humble/setup.sh' If installed via Jetbrain Toolbox, your pycharm.sh will be located at

'/home/{USER_NAME}/.local/share/JetBrains/Toolbox/apps/{pycharm_version}/bin/pycharm.sh'. For Jetbrain Remote Development, pycharm.sh is in

'/home/{USER_NAME}/.cache/JetBrains/RemoteDev/dist/{cached_pycharm_version}/bin/pycharm.sh'. The same applies to CLion and IntelliJ.

nixrunner commented 4 months ago

nixrunner@fedora:/usr/lib64/ros2-iron$ ros2 Traceback (most recent call last): File "/usr/lib64/ros2-iron/bin/ros2", line 33, in <module> sys.exit(load_entry_point('ros2cli==0.25.4', 'console_scripts', 'ros2')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/ros2-iron/bin/ros2", line 25, in importlib_load_entry_point return next(matches).load() ^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/importlib/metadata/__init__.py", line 205, in load module = import_module(match.group('module')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/ros2cli/cli.py", line 22, in <module> from rclpy.executors import ExternalShutdownException File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/__init__.py", line 49, in <module> from rclpy.signals import install_signal_handlers File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/signals.py", line 15, in <module> from rclpy.exceptions import InvalidHandle File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/exceptions.py", line 15, in <module> from rclpy.impl.implementation_singleton import rclpy_implementation as _rclpy File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/impl/implementation_singleton.py", line 32, in <module> rclpy_implementation = import_c_library('._rclpy_pybind11', package) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rpyutils/import_c_library.py", line 39, in import_c_library return importlib.import_module(name, package=package) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ImportError: librosidl_typesupport_c.so: cannot open shared object file: No such file or directory The C extension '/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/_rclpy_pybind11.cpython-312-x86_64-linux-gnu.so' failed to be imported while being present on the system. Please refer to 'https://docs.ros.org/en/{distro}/Guides/Installation-Troubleshooting.html#import-failing-even-with-library-present-on-the-system' for possible solutions

I am having a similar error. I faced with same error on Fedora 39 and Ubuntu 24.04 Noble

dheera commented 4 weeks ago

Bump, same error or Ubuntu 24.04 + rolling

Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rclpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/ros/rolling/lib/python3.12/site-packages/rclpy/__init__.py", line 49, in <module>
    from rclpy.signals import install_signal_handlers
  File "/opt/ros/rolling/lib/python3.12/site-packages/rclpy/signals.py", line 15, in <module>
    from rclpy.exceptions import InvalidHandle
  File "/opt/ros/rolling/lib/python3.12/site-packages/rclpy/exceptions.py", line 16, in <module>
    from rclpy.impl.implementation_singleton import rclpy_implementation as _rclpy
  File "/opt/ros/rolling/lib/python3.12/site-packages/rclpy/impl/implementation_singleton.py", line 32, in <module>
    rclpy_implementation = import_c_library('._rclpy_pybind11', package)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ros/rolling/lib/python3.12/site-packages/rpyutils/import_c_library.py", line 39, in import_c_library
    return importlib.import_module(name, package=package)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dheera/miniconda3/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'rclpy._rclpy_pybind11'
The C extension '/opt/ros/rolling/lib/python3.12/site-packages/_rclpy_pybind11.cpython-311-x86_64-linux-gnu.so' isn't present on the system. Please refer to 'https://docs.ros.org/en/rolling/How-To-Guides/Installation-Troubleshooting.html#import-failing-without-library-present-on-the-system' for possible solutions
NirvanaDragon commented 1 week ago

Bump, same error or Ubuntu 24.04 + rolling

Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rclpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/ros/rolling/lib/python3.12/site-packages/rclpy/__init__.py", line 49, in <module>
    from rclpy.signals import install_signal_handlers
  File "/opt/ros/rolling/lib/python3.12/site-packages/rclpy/signals.py", line 15, in <module>
    from rclpy.exceptions import InvalidHandle
  File "/opt/ros/rolling/lib/python3.12/site-packages/rclpy/exceptions.py", line 16, in <module>
    from rclpy.impl.implementation_singleton import rclpy_implementation as _rclpy
  File "/opt/ros/rolling/lib/python3.12/site-packages/rclpy/impl/implementation_singleton.py", line 32, in <module>
    rclpy_implementation = import_c_library('._rclpy_pybind11', package)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ros/rolling/lib/python3.12/site-packages/rpyutils/import_c_library.py", line 39, in import_c_library
    return importlib.import_module(name, package=package)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dheera/miniconda3/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'rclpy._rclpy_pybind11'
The C extension '/opt/ros/rolling/lib/python3.12/site-packages/_rclpy_pybind11.cpython-311-x86_64-linux-gnu.so' isn't present on the system. Please refer to 'https://docs.ros.org/en/rolling/How-To-Guides/Installation-Troubleshooting.html#import-failing-without-library-present-on-the-system' for possible solutions

You should check if the Python environment for ROS matches the current terminal's Python environment. I noticed your terminal is using Python 3.11, but ROS is looking for Python 3.12, which could be why it can't find the required library. This is just my guess.

Rizo11 commented 1 week ago

SOLUTION: I had the same problem. I was running it in conda env. I truned off conda env, and it worked!

dholukeval commented 17 hours ago

setcap solved the problem for me. here

sudo setcap -r /usr/bin/python3.10

This will remove extended capabilities form the executable. Change the python version according to your setup.