[BUG] PEP 668 breaks docker Python setup

[rcannood]

What happened?

The way we currently install Pypi packages no longer works with PEP668.

Steps to reproduce

name: test
arguments:
  - type: string
    name: --foo
    default: bar
resources:
  - type: python_script
    text: print(par)
test_resources:
  - type: bash_script
    text: "$meta_executable --foo zing"
engines:
  - type: docker
    image: ubuntu:24.04
    setup:
      - type: apt
        packages:
          - procps
          - python3-pip
      - type: python
        packages:
          - pandas

viash test config.vsh.yaml

#6 [3/3] RUN pip install --upgrade pip &&   pip install --upgrade --no-cache-dir "pandas"
#6 0.568 error: externally-managed-environment
#6 0.568 
#6 0.568 × This environment is externally managed
#6 0.568 ╰─> To install Python packages system-wide, try apt install
#6 0.568     python3-xyz, where xyz is the package you are trying to
#6 0.568     install.
#6 0.568     
#6 0.568     If you wish to install a non-Debian-packaged Python package,
#6 0.568     create a virtual environment using python3 -m venv path/to/venv.
#6 0.568     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
#6 0.568     sure you have python3-full installed.
#6 0.568     
#6 0.568     If you wish to install a non-Debian packaged Python application,
#6 0.568     it may be easiest to use pipx install xyz, which will manage a
#6 0.568     virtual environment for you. Make sure you have pipx installed.
#6 0.568     
#6 0.568     See /usr/share/doc/python3.12/README.venv for more information.
#6 0.568 
#6 0.568 note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
#6 0.568 hint: See PEP 668 for the detailed specification.
#6 ERROR: process "/bin/sh -c pip install --upgrade pip &&   pip install --upgrade --no-cache-dir \"pandas\"" did not complete successfully: exit code: 1
------
 > [3/3] RUN pip install --upgrade pip &&   pip install --upgrade --no-cache-dir "pandas":
0.568     sure you have python3-full installed.
0.568     
0.568     If you wish to install a non-Debian packaged Python application,
0.568     it may be easiest to use pipx install xyz, which will manage a
0.568     virtual environment for you. Make sure you have pipx installed.
0.568     
0.568     See /usr/share/doc/python3.12/README.venv for more information.
0.568 
0.568 note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
0.568 hint: See PEP 668 for the detailed specification.
------
Dockerfile:7
--------------------
   6 |     
   7 | >>> RUN pip install --upgrade pip && \
   8 | >>>   pip install --upgrade --no-cache-dir "pandas"
   9 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install --upgrade pip &&   pip install --upgrade --no-cache-dir \"pandas\"" did not complete successfully: exit code: 1
[error] Error occurred while building container '840419303237.dkr.ecr.us-east-1.amazonaws.com/viash_components/test:test'
ERROR! Setup failed!
Unexpected error occurred! If you think this is a bug, please post
create an issue at https://github.com/viash-io/viash/issues containing
a reproducible example and the stack trace below.

viash - 0.9.0-RC6
Stacktrace:
java.lang.RuntimeException: Setup failed!
        at io.viash.ViashTest$.apply(ViashTest.scala:137)
        at io.viash.Main$.mainCLI(Main.scala:256)
        at io.viash.Main$.mainCLIOrVersioned(Main.scala:124)
        at io.viash.Main$.main(Main.scala:59)
        at io.viash.Main.main(Main.scala)

Expected behavior

/

Relevant log output

No response

Version

Viash 0.9.0-RC6

Possible solution

No response

Confirmation

• [X] I have searched the existing issues to make sure this is not a duplicate.
• [X] I have provided clear and concise information about the bug.

Additional context

No response

[DriesSchaumont]

As a workaround, the following can be added to the docker engine setup:

      - type: docker
        env: PIP_BREAK_SYSTEM_PACKAGES=1