[Modular CLI]: modular install mojo fails behind proxy

esghif commented 7 months ago

Issue description

modular install mojo yields:

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1bb1c44580>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1bb1c2edc0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1bb0fa8280>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1bb0fa82e0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1bb0fa8c70>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
ERROR: Could not find a version that satisfies the requirement find_libpython==0.3.0 (from -r /root/.modular/pkg/packages.modular.com_mojo/scripts/post-install/requirements.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for find_libpython==0.3.0 (from -r /root/.modular/pkg/packages.modular.com_mojo/scripts/post-install/requirements.txt (line 1))
modular: error: failed to run python:

find_libpython does not exist in the venv installed by modular. It can be installed by pip which correctly uses environment variables http_proxy and https_proxy. AFAICT the installation process tries to install using python urllib3 library which does not use the proxy environment variables.

Steps to reproduce

root@BP-XXXXX:/# modular install mojo
# Found release for https://packages.modular.com/mojo @ 0.5.0
# Installing to /root/.modular/pkg/packages.modular.com_mojo
# Downloading artifacts. Please wait...
# Downloads complete, setting configs...
# Configs complete, running post-install hooks...
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f4bee6a2580>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f4bee6b2dc0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f4bed340280>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f4bed3402e0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f4bed340c70>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/find-libpython/
ERROR: Could not find a version that satisfies the requirement find_libpython==0.3.0 (from -r /root/.modular/pkg/packages.modular.com_mojo/scripts/post-install/requirements.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for find_libpython==0.3.0 (from -r /root/.modular/pkg/packages.modular.com_mojo/scripts/post-install/requirements.txt (line 1))
modular: error: failed to run python:

root@BP-XXXXX:/# . /root/.modular/pkg/packages.modular.com_mojo/venv/bin/activate
(venv) root@BP-X035448:/# pip install find_libpython
Collecting find_libpython
  Using cached find_libpython-0.3.1-py3-none-any.whl (8.4 kB)
Installing collected packages: find-libpython
Successfully installed find-libpython-0.3.1

Version Info

- Provide Modular CLI version by pasting the output of `modular -v`
modular 0.2.2 (95d42445)
- What OS did you install modular CLI on ?
Ubuntu 20.04.6 LTS

esghif commented 7 months ago

I was able to circumvent this issue by doing the following in the Dockerfile:

RUN modular auth $AUTH_KEY \
    && modular install mojo; \
    exit 0
RUN rm /root/.modular/pkg/packages.modular.com_mojo/lib/liblldb.so \
    /root/.modular/pkg/packages.modular.com_mojo/lib/libMojoJupyter.so \
    /root/.modular/pkg/packages.modular.com_mojo/lib/liblldb.so.18git \
    /root/.modular/pkg/packages.modular.com_mojo/lib/libKGENCompilerRTShared.so \
    /root/.modular/pkg/packages.modular.com_mojo/lib/libMojoLLDB.so \
    /root/.modular/pkg/packages.modular.com_mojo/lib/libLLCLRuntimeGlobals.so \
    /root/.modular/pkg/packages.modular.com_mojo/lib/libMSupportGlobals.so
RUN . /root/.modular/pkg/packages.modular.com_mojo/venv/bin/activate \
    && pip install -r /root/.modular/pkg/packages.modular.com_mojo/scripts/post-install/requirements.txt \
    && deactivate
RUN modular install mojo

A nicer solution would be preferable 😉

adah1972 commented 3 months ago

Yup, modular install seems to ignore https_proxy settings etc.

Thank you for the workaround. It has helped.

modularml / mojo