search

p4lang / ptf

Packet Test Framework
Apache License 2.0
144 stars 99 forks source link

Suggestions for eventually updating where Python3 packages are installed via pip? #198

Open jafingerhut opened 1 year ago

jafingerhut commented 1 year ago

I have been doing some early experiments with testing my install script that installs a bunch of P4 open source development tools with Ubuntu 23.04, and found that when you attempt to run a command like sudo pip install some-python-package, it gives the following error message:

$ sudo pip install ply
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

I believe this is because Ubuntu 23.04 has a newer version of pip installed by default than earlier Ubuntu versions, and this newer version of pip gives an error in this situation, not merely a warning as previous versions did.

Potentially the least-effort way to hack around this would be to add --break-system-packages to all sudo pip install ... commands, across all p4lang projects, and any dependencies they install.

The way recommended by Python developers appears to be to use a Python virtual environment and use pip to install packages into that. That also seems like it might require updates to all sudo pip install ... commands across p4lang projects to remove the sudo, as well as a common convention for assuming that a virtual env has been set up before their install scripts are run? Not sure.

antoninbas commented 1 year ago

It looks like the same error occurs when using pip install --user ... instead of sudo pip install .... IMO the best option would be to use pipx as suggested, or a virtual env that you manage directly. Or you can just export PIP_BREAK_SYSTEM_PACKAGES=1 in the correct profile file, and then forget about it. You won't have to use --break-system-packages explicitly every time, and you can keep using pip as before.

jafingerhut commented 1 year ago

I did not know about the environment variable PIP_BREAK_SYSTEM_PACKAGES.

From some experimentation on an Ubuntu 23.04 system, it seems that only works if I use sudo -E pip install ..., so it is just as troublesome in requiring that all install scripts are edited, as any of the other methods you mention.

I may see what it takes to remove the sudo from all of the sudo pip3 ... commands everywhere from all p4lang install scripts in my local copies -- there probably aren't more than half a dozen or so occurrences.