Closed marcoag closed 4 months ago
I'm willing to bet that this is a regression caused by #1026. The distutils
module is normally provided by setuptools.
Also, somewhat ironically, this doesn't appear to fix the problem. In a fresh noble VM, the python3-distutils
package appears to install the module for Python 3.11, not 3.12 as is required.
Oh, this is fun, I think I figured it out.
python3-colcon-python-setup-py
has a bug and is missing a runtime dependency on python3-setuptools
. I'd suppose we should file a ticket.All three factors were needed for this behavior to manifest.
My recommendation is to install python3-setuptools
here instead of python3-distutils
. There are other errors in the job referring to setuptools, so even if python3-distutils
wasn't bugged on Noble, we'd still need setuptools installed. After we resolve (1), we can revert this change.
My recommendation is to install
python3-setuptools
here instead ofpython3-distutils
. There are other errors in the job referring to setuptools, so even ifpython3-distutils
wasn't bugged on Noble, we'd still need setuptools installed. After we resolve (1), we can revert this change.
I think that in the end we will likely need both the apt setuptools and distutils packages since Debian splits the modules into separate even though they're both part of setuptools upstream.
python3-setuptools
seems to have a dependency on python3-distutils
though:
$ apt-cache depends python3-setuptools
python3-setuptools
Depends: python3-pkg-resources
Depends: python3-distutils
Depends: <python3:any>
python3:i386
python3
Suggests: python-setuptools-doc
I've put up #1028 which restores @marcoag's original proposal while we figure out the rest of what should happen in order to remove this pip installation of setuptools.
python3-setuptools
seems to have a dependency onpython3-distutils
though:
It's inconsequential. I have no idea why they did that.
Installing python3-distutils
alone doesn't work for Python 3.12:
$ sudo apt install -y python3-distutils
...
$ python3 -c "import distutils; print(distutils.__file__)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'distutils'
It does, however, work for Python 3.11:
$ python3.11 -c "import distutils; print(distutils.__file__)"
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
/usr/lib/python3.11/distutils/__init__.py
The python3-setuptools
package bundles distutils and uses a .pth
file to inject it into the Python module search:
$ dpkg -L python3-setuptools
...
/usr/lib/python3/dist-packages
/usr/lib/python3/dist-packages/_distutils_hack
/usr/lib/python3/dist-packages/_distutils_hack/__init__.py
/usr/lib/python3/dist-packages/_distutils_hack/override.py
/usr/lib/python3/dist-packages/distutils-precedence.pth
...
/usr/lib/python3/dist-packages/setuptools/_distutils
/usr/lib/python3/dist-packages/setuptools/_distutils/__init__.py
...
$ python3 -c "import distutils; print(distutils.__file__)"
/usr/lib/python3/dist-packages/setuptools/_distutils/__init__.py
tl;dr - Ignore Ubuntu's python3-distutils
package entirely, and assume that python3-setuptools
provides the mechanism to import distutils
. I do not believe that installing setuptools from pip is necessary, @nuclearsandwich.
In a maintenance meeting, @cottsay and I have discussed the various fixes here: unconditionally installing setuptools, installing setuptools alongside the colcon package with the missing dependency, and since this is all arising because of the colcon version in upstream noble missing a runtime dependency on setuptools, updating our version of colcon-core so it has higher precedence than the upstream one again: That's being handled in https://github.com/ros-infrastructure/reprepro-updater/pull/203 which is blocked on a deployment issue but moving forward. Let's hold off on either this PR or #1028 until that lands to see if it fixes this issue.
see if it fixes this issue
Alright, I can now see python3-setuptools
getting installed with colcon, but we finally hit the wall on our conflicts with the upstream colcon packages and we need to address that to move forward.
At present, our python3-colcon-core
provides /usr/bin/colcon
, which is provided by a package simply called colcon
in upstream Debian. Without any metadata which indicates that they conflict, both packages are being installed and the installation fails during unpacking. Even if we have a pinning solution to make apt prefer our packages over upstream's, we should not allow apt to even try to install them both if we know the operation will fail.
I'm proposing that our python3-colcon-core
"Replaces" the colcon
package from upstream: colcon/colcon-core#628
This should allow the installation of upstream's colcon
package to proceed, but the conflicting files will come from our python3-colcon-core
automatically.
The aforementioned "Replaces" change to colcon-core
has resolved the issue, and Rdev/Rpr jobs appear to be turning over.
Dev jobs are failing like: https://build.ros2.org/view/Rdev/job/Rdev__rqt_srv__ubuntu_noble_amd64/2/console with:
This PR adds
python3-distutils
to thedevel_task.Dockerfile.em
file in an effort to fix the missing dependency.