Unable to build container when host is using pip >= 23.0

[nosseb]

Checklist

• [X] I've read the contribution guidelines.
• [X] I've searched other issues and no duplicate issues were found.
• [X] I'm convinced that this is not my fault but a bug.

Description

When building the container (./setup-dev-env.sh -y docker) on a host using pip >= 23.0.1 (e.g. Ubuntu >= 23.10), the build fail on the install of pipx with error: externally-managed-environment.

Expected behavior

No error when building the container ; successful build.

Actual behavior

Failed build :

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Steps to reproduce

1. Use pip >= 23.0 on the host
2. Download the repo with git clone https://github.com/autowarefoundation/autoware.git
3. cd autoware
4. Attempt to build the container with ./setup-dev-env.sh -y --no-nvidia docker
5. Wait for the error to appear

Versions

• OS : Ubuntu 24.04 (probably also on 23.10, untested)
• pip : 24.0 (but since 23.0 according to https://pip.pypa.io/en/stable/news/#v23-0)
• autoware : latest commit (886409979b78019e75185a0b57e11cb1b449942b)

Possible causes

PEP 668 (May 2021) marks Python default environment as "externally managed", making it so that one can't use pip to install packages in the default env (as performed in setup-dev-env.sh on line 182 :

# Install pipx for ansible
if ! (python3 -m pipx --version >/dev/null 2>&1); then
    sudo apt-get -y update
    python3 -m pip install --user pipx
fi

Additional context

No response

[nosseb]

Possibles fixes :

1. use the --break-system-packages option.

Pros :

• easy

Cons :

• not supported by pip <= 23.0.1 (https://pip.pypa.io/en/stable/news/#v23-0-1)
• might break the user's python install

2. use the PIP_BREAK_SYSTEM_PACKAGES environement variables

Pros :

• probably ignored by older versions of pip

  Cons :

• might break the user's python install

3. install pipx via apt

Pros :

• cleanest option

  Cons :

• pipx is only available in apt for Ubuntu >= 20 (https://packages.ubuntu.com/search?keywords=pipx&searchon=names&suite=all&section=all)

edit : formating edit2 : for reference, Ubuntu 18.04 LTS standard security maintenance ended in 2023