search

tnc-br / ddf-isoscapes

4 stars 0 forks source link

`pip install` in local runtime fails in with `error:externally-managed-environment` #69

Closed jmogarrio closed 1 year ago

jmogarrio commented 1 year ago

Ran into this issue trying to start a Jupyter server, tornado was stuck in an older version. Mitigated by using --break-system-packages, as in:

pip install --break-system-packages --force-reinstall tornado

Not sure that we want to leave that in the code.

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.
jmogarrio commented 1 year ago

Forgot to do the pull request but I think this is mitigated with this commit.

jmogarrio commented 1 year ago

For future reference, this is what I tried. I was working with a requirements.txt file, which I might try again, but pipx seemed like the fastest fix.

## pipx approach

!pipx install requirements.txt

Fatal error from pip prevented installation. Full pip output in file:
    /usr/local/google/home/jmogarrio/.local/pipx/logs/cmd_2023-06-15_11.20.17_pip_errors.log

Some possibly relevant errors from pip install:
    ERROR: Could not find a version that satisfies the requirement requirements-txt (from versions: none)
    ERROR: No matching distribution found for requirements-txt

Error installing requirements-txt.
## pip -r requirements.txt approach

!pip install -r --break-system-packages requirements.txt

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

I tried different combinations of pip install gdal and pipx install gdal, but the GDAL package doc itself mentions that installation can be tricky, so this might warrant specialized effort.

Wanted to get rid of the sudo apt-get install python3 sklearn, but since right now I can't do dev runs of the pipeline I'm worried attempting that via pipx install sklearn might break something.

jmogarrio commented 1 year ago

Breaking down "Make package installation uniform and robust.", which is now mostly done through the listed pull requests. Remaining work for that item is in #92, which no longer affects this issue.