"cannot import name 'generate_py' from 'rosidl_generator_py'" when going through workspace tutorial

[aliddell]

Bug report

Required Info:

• Operating System:

  • Ubuntu 22.04

• Installation type:

  • Binaries

• Version or commit hash:

  • (I'm guessing this is the relevant package:
  • Package: ros-rolling-rosidl-generator-py
  • Version: 0.15.0-1jammy.20220521.000012

• DDS implementation:

  • (Guessing) Fast-RTPS.

• Client library (if applicable):

  • Not sure if applicable.

Steps to reproduce issue

Run through this tutorial. I'm observing this in rolling and humble.

$ source /opt/ros/rolling/setup.bash
$ mkdir -p ~/dev_ws/src && cd ~/dev_ws/src
$ git clone https://github.com/ros/ros_tutorials.git -b rolling-devel
$ cd ~/dev_ws && rosdep install -i --from-path src --rosdistro rolling -y
$ colcon build

Expected behavior

Turtlesim builds.

Actual behavior

Receive the following output:

[0.270s] WARNING:colcon.colcon_core.package_selection:Some selected packages are already built in one or more underlay workspaces:
    'turtlesim' is in: /opt/ros/rolling
If a package in a merged underlay workspace is overridden and it installs headers, then all packages in the overlay must sort their include directories by workspace order. Failure to do so may result in build failures or undefined behavior at run time.
If the overridden package is used by another package in any underlay, then the overriding package in the overlay must be API and ABI compatible or undefined behavior at run time may occur.

If you understand the risks and want to override a package anyways, add the following to the command line:
    --allow-overriding turtlesim

This may be promoted to an error in a future release of colcon-override-check.
Starting >>> turtlesim
--- stderr: turtlesim                             
Traceback (most recent call last):
  File "/opt/ros/rolling/share/rosidl_generator_py/cmake/../../../lib/rosidl_generator_py/rosidl_generator_py", line 8, in <module>
    from rosidl_generator_py import generate_py
ImportError: cannot import name 'generate_py' from 'rosidl_generator_py' (/opt/ros/rolling/local/lib/python3.10/dist-packages/rosidl_generator_py/__init__.py)
gmake[2]: *** [turtlesim__py/CMakeFiles/turtlesim__py.dir/build.make:121: rosidl_generator_py/turtlesim/_turtlesim_s.ep.rosidl_typesupport_fastrtps_c.c] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:706: turtlesim__py/CMakeFiles/turtlesim__py.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
gmake: *** [Makefile:146: all] Error 2
---
Failed   <<< turtlesim [0.33s, exited with code 2]

Summary: 0 packages finished [0.45s]
  1 package failed: turtlesim
  1 package had stderr output: turtlesim

Additional information

If this is the wrong place to report this error, do point me in the right direction.

[clalancette]

Hm, that's odd. I can't reproduce the problem here; the turtlesim package builds fine for me.

Can you go into a fresh terminal (where you haven't sourced anything ROS related), and then do:

source /opt/ros/rolling/setup.bash
python3 -c 'from rosidl_generator_py import generate_py'

I'm assuming that is going to fail as well. If it does, I think it bears looking into the contents of /opt/ros/rolling/local/lib/python3.10/dist-packages/rosidl_generator_py/generate_py_impl.py to ensure that the generate_py method exists in that file, as that is where it should be coming from.

[aliddell]

Yeah, it fails:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: cannot import name 'generate_py' from 'rosidl_generator_py' (/opt/ros/rolling/local/lib/python3.10/dist-packages/rosidl_generator_py/__init__.py)

I haven't seen anyone else come across this problem either, so there's probably something wrong with my setup, but it's a fresh install of rolling. I see generate_py on line 56 of /opt/ros/rolling/local/lib/python3.10/dist-packages/rosidl_generator_py/generate_py_impl.py. Looking a little deeper,

  File "/opt/ros/rolling/share/rosidl_generator_py/cmake/../../../lib/rosidl_generator_py/rosidl_generator_py"

points to /opt/ros/rolling/lib/rosidl_generator_py/rosidl_generator_py. There's nothing else in that directory. Here's the script:

#!/usr/bin/env python3

import argparse
import os
import sys

try:
    from rosidl_generator_py import generate_py
except ImportError:
    # modifying sys.path and importing the Python package with the same
    # name as this script does not work on Windows
    rosidl_generator_py_root = os.path.dirname(os.path.dirname(__file__))
    rosidl_generator_py_module = os.path.join(
        rosidl_generator_py_root, 'rosidl_generator_py', '__init__.py')
    if not os.path.exists(rosidl_generator_py_module):
        raise
    from importlib.machinery import SourceFileLoader

    loader = SourceFileLoader('rosidl_generator_py', rosidl_generator_py_module)
    rosidl_generator_py = loader.load_module()
    generate_py = rosidl_generator_py.generate_py

def main(argv=sys.argv[1:]):
    parser = argparse.ArgumentParser(description='Generate the Python ROS interfaces.')
    parser.add_argument(
        '--generator-arguments-file',
        required=True,
        help='The location of the file containing the generator arguments')
    parser.add_argument(
        '--typesupport-impls',
        required=True,
        help='All the available typesupport implementations')
    args = parser.parse_args(argv)

    generate_py(args.generator_arguments_file, args.typesupport_impls.split(';'))
    return 0

if __name__ == '__main__':
    sys.exit(main())

I'm assuming the ImportError gets triggered, so when we get to the following:

rosidl_generator_py_root = os.path.dirname(os.path.dirname(__file__))
rosidl_generator_py_module = os.path.join(
    rosidl_generator_py_root, 'rosidl_generator_py', '__init__.py')

rosidl_generator_py_root actually points to /opt/ros/rolling/lib, so that rosidl_generator_py_module is looking for /opt/ros/rolling/lib/rosidl_generator_py/__init__.py, which doesn't exist (because the only file in that directory is the script above). So I'm not sure if this is an issue in packaging or if there were some symlinks that should have been created and weren't, or what.

Happy to provide any more details you might need.

[clalancette]

rosidl_generator_py_root actually points to /opt/ros/rolling/lib, so that rosidl_generator_py_module is looking for /opt/ros/rolling/lib/rosidl_generator_py/__init__.py, which doesn't exist (because the only file in that directory is the script above). So I'm not sure if this is an issue in packaging or if there were some symlinks that should have been created and weren't, or what.

Yeah, that doesn't exist, but it shouldn't need to. That fallback should only ever be used on Windows. The real thing that is failing is from rosidl_generator_py import generate_py, which should be succeeding.

I just tried a fresh installation of Jammy and Rolling following the directions at https://docs.ros.org/en/rolling/Installation/Ubuntu-Install-Debians.html , and I'm able to successfully import:

$ source /opt/ros/rolling/setup.bash
$ python3 -c 'from rosidl_generator_py import generate_py'
$

What were the exact steps you used to install Rolling?

Separately, are you maybe using a python virtual environment or something like that?

[aliddell]

What were the exact steps you used to install Rolling?

I followed the same instructions.

Separately, are you maybe using a python virtual environment or something like that?

I do have Anaconda sourced, now you mention it. If I

$ conda deactivate
$ python3 -c 'from rosidl_generator_py import generate_py'

I'm successful. Also, if I don't activate Anaconda and run through the tutorial steps above, I can successfully build turtlesim. Thanks for helping me through this!

[sloretz]

I believe humble packages are supposed to be built with python3.6

On Ubuntu Jammy (22.04) Humble uses Python 3.10. See REP-2000.

[IamPhytan]

Hello, I have the same problem as @aliddell, without any Python/Conda environment that could have an effect on the import

It works when I import it simply with python3, but it doesn't work when I colcon build.

user@hostname:~/tutorial_ws$ python3 -c 'from rosidl_generator_py import generate_py'
user@hostname:~/tutorial_ws$ colcon build --packages-select tutorial_interfaces 
Starting >>> tutorial_interfaces
--- stderr: tutorial_interfaces                             
Traceback (most recent call last):
  File "/opt/ros/humble/share/rosidl_generator_py/cmake/../../../lib/rosidl_generator_py/rosidl_generator_py", line 8, in <module>
    from rosidl_generator_py import generate_py
ImportError: cannot import name 'generate_py'
gmake[2]: *** [tutorial_interfaces__py/CMakeFiles/tutorial_interfaces__py.dir/build.make:157: rosidl_generator_py/tutorial_interfaces/_tutorial_interfaces_s.ep.rosidl_typesupport_fastrtps_c.c] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:590: tutorial_interfaces__py/CMakeFiles/tutorial_interfaces__py.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
gmake: *** [Makefile:146: all] Error 2
---
Failed   <<< tutorial_interfaces [1.89s, exited with code 2]

Summary: 0 packages finished [2.07s]
  1 package failed: tutorial_interfaces
  1 package had stderr output: tutorial_interfaces

My current workaround is to comment out this line in the CMakeLists.txt :

# find_package(rosidl_default_generators REQUIRED)

[mrayh001]

Hi, I am also facing the same issue with ros2 humble and ubuntu 22.04

when i do python3 -c 'from rosidl_generator_py import generate_py' I can import. I can also import with python Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

from rosidl_generator_py import generate_py

But when I do colcon build it says Traceback (most recent call last): File "/opt/ros/humble/share/rosidl_generator_py/cmake/../../../lib/rosidl_generator_py/rosidl_generator_py", line 8, in from rosidl_generator_py import generate_py ImportError: cannot import name 'generate_py' gmake[2]: [spot_msgs__py/CMakeFiles/spot_msgs__py.dir/build.make:1497: rosidl_generator_py/spot_msgs/_spot_msgs_s.ep.rosidl_typesupport_fastrtps_c.c] Error 1 gmake[1]: [CMakeFiles/Makefile2:590: spot_msgs__py/CMakeFiles/spot_msgs__py.dir/all] Error 2 gmake: *** [Makefile:146: all] Error 2

Failed <<< spot_msgs [43.2s, exited with code 2]