Closed BrunoB81HK closed 10 months ago
@BrunoB81HK Which version of Ubuntu and ROS 2 are you using? I can build with --symlink-install
and run the example on this image no problem.
I'm using humble with Ubuntu 22.04. We are also using docker and our container is generated from this one.
Are you building with any extra options? When I run colcon build --symlink-install
to build and ros2 run generate_parameter_module_example test_node --ros-args --params-file src/generate_parameter_library/example_python/config/implementation.yaml
to run, it works even on the image you mentioned.
OK, that is weird. The only other that we use is the following --cmake-args -DCMAKE_BUILD_TYPE=Release
. Maybe the error is on my side.
Okay, that is strange. The only other thing I'd suggest is to ensure that rosdep is run: rosdep install --from-paths src --ignore-src -y
before building and running.
rosdep install --from-paths src --ignore-src -y
is indeed called before building and running.
For my custom package, python can find it, but I think its name collide with another package so that parameters.py
is not found.
I compiled the differences between a code built with and without --symink-install
below.
In my case, using --symink-install
cause a duplicated module in /colcon_ws/build
that is added in the python path and, since this module is a symlink to the /colcon_ws/src
, it doesn't contain the generated parameters.py
.
The generated parameters.py
is theoretically reachable from /home/user/colcon_ws/install/pkg_name/lib/python3.10/site-packages
, but is hidden behind the one in /home/user/colcon_ws/build/pkg_name
.
The most surprising thing to me is that it happens to me but not to you. @pac48, do you have any idea why this happens and how I could remedy this problem? Is it a problem with my setup? Or a problem with my module naming?
--symlink-install
__file__
):/home/user/colcon_ws/build/pkg_name/pkg_name/script.py
sys.path
):Paths |
---|
1️⃣ /home/user/colcon_ws/install/pkg_name/lib/pkg_name |
2️⃣ /home/user/colcon_ws/build/pkg_name |
3️⃣ /home/user/colcon_ws/install/pkg_name/lib/python3.10/site-packages |
/opt/ros/humble/lib/python3.10/site-packages |
/opt/ros/humble/local/lib/python3.10/dist-packages |
/usr/lib/python310.zip |
/usr/lib/python3.10 |
/usr/lib/python3.10/lib-dynload |
/usr/local/lib/python3.10/dist-packages |
/usr/lib/python3/dist-packages |
sys.modules
):Module | Location |
---|---|
pkg_name.script | 2️⃣ /home/user/colcon_ws/build/pkg_name |
pkg_name | 2️⃣ /home/user/colcon_ws/build/pkg_name |
colcon_ws
├── build
│ ├── COLCON_IGNORE
│ └── 2️⃣ pkg_name
│ ├── pkg_name -> /home/user/colcon_ws/src/pkg_name/pkg_name
│ ├── pkg_name.egg-info
│ │ └── ...
│ ├── package.xml -> /home/user/colcon_ws/src/pkg_name/package.xml
│ ├── resource
│ │ └── pkg_name -> /home/user/colcon_ws/src/pkg_name/resource/pkg_name
│ ├── setup.cfg -> /home/user/colcon_ws/src/pkg_name/setup.cfg
│ ├── setup.py -> /home/user/colcon_ws/src/pkg_name/setup.py
│ ├── ...
│ └── share
│ └── pkg_name
│ └── hook
│ ├── pythonpath_develop.dsv
│ ├── pythonpath_develop.ps1
│ └── pythonpath_develop.sh
├── install
│ ├── COLCON_IGNORE
│ ├── pkg_name
│ │ ├── lib
│ │ │ ├── 1️⃣ pkg_name
│ │ │ │ └── entry_point
│ │ │ └── python3.10
│ │ │ └── 3️⃣ site-packages
│ │ │ ├── pkg-name.egg-link
│ │ │ └── pkg_name
│ │ │ └── parameters.py
│ │ └── share
│ │ ├── ...
│ │ └── colcon-core
│ │ └── packages
│ │ └── pkg_name
│ └── ...
└── src
└── pkg_name
├── pkg_name
│ ├── __init__.py
│ ├── script.py
│ └── parameters.yaml
├── package.xml
├── resource
│ └── pkg_name
├── setup.cfg
└── setup.py
--symlink-install
__file__
):/home/user/colcon_ws/install/pkg_name/lib/python3.10/site-packages/pkg_name/script.py
sys.path
):Paths |
---|
1️⃣ /home/user/colcon_ws/install/pkg_name/lib/pkg_name |
2️⃣ /home/user/colcon_ws/install/pkg_name/lib/python3.10/site-packages |
/opt/ros/humble/lib/python3.10/site-packages |
/opt/ros/humble/local/lib/python3.10/dist-packages |
/usr/lib/python310.zip |
/usr/lib/python3.10 |
/usr/lib/python3.10/lib-dynload |
/usr/local/lib/python3.10/dist-packages |
/usr/lib/python3/dist-packages |
sys.modules
):Module | Location |
---|---|
pkg_name.script | 2️⃣ /home/user/colcon_ws/install/pkg_name/lib/python3.10/site-packages |
pkg_name | 2️⃣ /home/user/colcon_ws/install/pkg_name/lib/python3.10/site-packages |
colcon_ws
├── build
│ ├── COLCON_IGNORE
│ └── pkg_name
│ ├── pkg_name
│ │ └── parameters.py
│ ├── pkg_name.egg-info
│ │ └── ...
│ ├── build
│ │ └── lib
│ │ └── pkg_name
│ │ ├── __init__.py
│ │ └── script.py
│ └── ...
├── install
│ ├── COLCON_IGNORE
│ ├── pkg_name
│ │ ├── lib
│ │ │ ├── 1️⃣ pkg_name
│ │ │ │ └── entry_point
│ │ │ └── python3.10
│ │ │ └── 2️⃣ site-packages
│ │ │ ├── pkg_name-0.1.0-py3.10.egg-info
│ │ │ │ └── ...
│ │ │ └── pkg_name
│ │ │ ├── __init__.py
│ │ │ ├── __pycache__
│ │ │ │ └── ...
│ │ │ ├── script.py
│ │ │ └── parameters.py
│ │ └── share
│ │ ├── ...
│ │ └── colcon-core
│ │ └── packages
│ │ └── pkg_name
│ └── ...
└── src
└── pkg_name
├── pkg_name
│ ├── __init__.py
│ ├── script.py
│ └── parameters.yaml
├── package.xml
├── resource
│ └── pkg_name
├── setup.cfg
└── setup.py
colcon_ws
├── build
│ ├── COLCON_IGNORE
│ └── pkg_name
- │ ├── pkg_name
- │ │ └── parameters.py
+ │ ├── pkg_name -> /home/user/colcon_ws/src/pkg_name/pkg_name
│ ├── pkg_name.egg-info
│ │ └── ...
- │ ├── build
- │ │ └── lib
- │ │ └── pkg_name
- │ │ ├── __init__.py
- │ │ └── script.py
+ │ ├── package.xml -> /home/user/colcon_ws/src/pkg_name/package.xml
+ │ ├── resource
+ │ │ └── pkg_name -> /home/user/colcon_ws/src/pkg_name/resource/pkg_name
+ │ ├── setup.cfg -> /home/user/colcon_ws/src/pkg_name/setup.cfg
+ │ ├── setup.py -> /home/user/colcon_ws/src/pkg_name/setup.py
+ │ └── share
+ │ └── pkg_name
+ │ └── hook
+ │ ├── pythonpath_develop.dsv
+ │ ├── pythonpath_develop.ps1
+ │ └── pythonpath_develop.sh
├── install
│ ├── COLCON_IGNORE
│ ├── pkg_name
│ │ ├── lib
│ │ │ ├── pkg_name
│ │ │ │ └── entry_point
│ │ │ └── python3.10
│ │ │ └── site-packages
- │ │ │ ├── pkg_name-0.1.0-py3.10.egg-info
- │ │ │ │ └── ...
+ │ │ │ ├── pkg-name.egg-link
│ │ │ └── pkg_name
- │ │ │ ├── __init__.py
- │ │ │ ├── __pycache__
- │ │ │ │ └── ...
- │ │ │ ├── script.py
│ │ │ └── parameters.py
│ │ └── share
│ │ ├── ...
│ │ └── colcon-core
│ │ └── packages
│ │ └── pkg_name
│ └── ...
└── src
└── pkg_name
├── pkg_name
│ ├── __init__.py
│ ├── script.py
│ └── parameters.yaml
├── package.xml
├── resource
│ └── pkg_name
├── setup.cfg
└── setup.py
@pac48 is there any way you can share me the Dockerfile
you used that worked for you?
@BrunoB81HK this is the image that I used. Its called osrf/ros:humble-desktop-full
After much investigating, I found out that the probleme lied in my Dockerfile configuration. The build process was not saving any changes made inside the src/
directory. I'll close this issue since it is not a bug. Thanks a lot for your help!
@BrunoB81HK can you share the changes you made to fix the problem? I have exactly the same problem with --symlink-install
@ahi-bplus Are you building the project inside your Dockerfile
? If so, the problem for me was pretty simple.
When using --symlink-install
, it creates a new module inside the src/
directory. The module is accessible because, if I remember correctly, the whole python module directory (the one who contains the __init__.py
file) is symlinked, not just the files. This is not a problem in itself, but becomes one if you mount your src/
into the container. If you do so, the module inside the install/
directory changes to the one mounted in your src/
directory. For that reason, the parameter module is overwritten.
I can give you some ideas to help with your case:
--symlink-install
inside the Dockerfile
.Dockerfile
.src/
directory inside the container.I hope it helps, feel free to ask more questions if you need! :)
@BrunoB81HK Thanks for your quick reply. I already use a devcontainer for development. I mount my src files accordingly. As soon as I build my ROS node which implements generate_parameter_library
using colcon build --symlink-install --merge-install
, I get the problem with ModuleNotFoundError
. Without --symlink-install --merge-install
it works.
Okay, I have found the problem. It has nothing to do with --symlink-install
or --merge-install
. The problem comes from my package structure.
package_name/
├── README.md
├── __init__.py
├── config
│ └── default
│ └── default.yaml
├── models
├── package.xml
├── requirements.txt
├── resource
│ └── package_name
├── setup.cfg
├── setup.py
├── src
│ └── package_name
│ ├── __init__.py
│ ├── node.py
│ ├── parameters.yaml
Unfortunately the package_dir={'': 'src'},
of generate_parameter_library
is not considered. If I avoid the subdirecory src/package_name
everything works. I think this would be worth an issue?
If you think this is a bug, you should create a new issue. Even if it's not, that will give others that have the same problem some help too.
When building the
example_python
package inside my workspace, I get theModuleNotFoundError
. This happens everytime I try to build with--symlink-install
. I first encountered this bug with my package and I decided to try the example directly to see if it happened with it too and it does. here is the traceback:I also checked my
PYTHONPATH
and I think that it may be related to name collisions.I saw that @pac48 was referencing to some side-effects when using
--symlink-install
in #122, is this related?