mamba-org / mamba

The Fast Cross-Platform Package Manager
https://mamba.readthedocs.io
BSD 3-Clause "New" or "Revised" License
6.57k stars 344 forks source link

[BUG] NO_PROXY not working with mamba #2105

Closed usinelogicielle closed 1 year ago

usinelogicielle commented 1 year ago

Hello,

Problems : We use mamba in our CI to download packages in our artifactory instance . We configure our enterprise proxy to access to internet and specify NO_PROXY and no_proxy with . mamba try to use the proxy and failed with :

RuntimeError: Download error (28) Timeout was reached [https://<login>:<password>@<artifactory.domains.com>/artifactory/api/conda/conda/noarch/repodata.json]
    SSL connection timeout

No problem using conda

Expected behaviour :

mamba doesn't use the proxy for URL with because no_proxy is defined

Tested with 0.18 to 1.0.0 version of mamba and conda 4.9.0 and 22.9.0 : All <> informations in log are sanitized info.

Here the base command :

export MAMBA_VERSION=0.19 && conda create --name mamba_$MAMBA_VERSION  mamba==$MAMBA_VERSION --yes && conda activate mamba_$MAMBA_VERSION && CONDA_VERBOSITY=2 mamba create --yes --name mamba_install_$MAMBA_VERSION python=3.9 numpy

We try to add CURLOPT_NOPROXY env variable after this explaination withtout success : https://curl.se/libcurl/c/CURLOPT_NOPROXY.html

Complete failed with last version :

LOGS HERE

``` $export MAMBA_VERSION=1.0 && conda create --name mamba_$MAMBA_VERSION mamba==$MAMBA_VERSION --yes && conda activate mamba_$MAMBA_VERSION && CONDA_VERBOSITY=2 mamba create --yes --name mamba_install_$MAMBA_VERSION python=3.9 numpy Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: done ## Package Plan ## environment location: /home//scratch/.conda/envs/mamba_1.0 added / updated specs: - mamba==1.0 The following NEW packages will be INSTALLED: _libgcc_mutex conda/linux-64::_libgcc_mutex-0.1-conda_forge None _openmp_mutex conda/linux-64::_openmp_mutex-4.5-2_gnu None brotlipy conda/linux-64::brotlipy-0.7.0-py310h5764c6d_1005 None bzip2 conda/linux-64::bzip2-1.0.8-h7f98852_4 None c-ares conda/linux-64::c-ares-1.18.1-h7f8727e_0 None ca-certificates conda/linux-64::ca-certificates-2022.10.11-h06a4308_0 None certifi conda/linux-64::certifi-2022.9.24-py310h06a4308_0 None cffi conda/linux-64::cffi-1.15.1-py310h255011f_2 None charset-normalizer conda/noarch::charset-normalizer-2.1.1-pyhd8ed1ab_0 None conda conda/linux-64::conda-22.9.0-py310hff52083_2 None conda-package-han~ conda/linux-64::conda-package-handling-1.9.0-py310h5eee18b_1 None cryptography conda/linux-64::cryptography-38.0.3-py310h600f1e7_0 None fmt conda/linux-64::fmt-9.1.0-h924138e_0 None icu conda/linux-64::icu-70.1-h27087fc_0 None idna conda/linux-64::idna-3.4-py310h06a4308_0 None keyutils conda/linux-64::keyutils-1.6.1-h166bdaf_0 None krb5 conda/linux-64::krb5-1.19.3-h08a2579_0 None ld_impl_linux-64 conda/linux-64::ld_impl_linux-64-2.39-hc81fddc_0 None libarchive conda/linux-64::libarchive-3.5.2-hada088e_3 None libcurl conda/linux-64::libcurl-7.86.0-h2283fc2_1 None libedit conda/linux-64::libedit-3.1.20210910-h7f8727e_0 None libev conda/linux-64::libev-4.33-h7f8727e_1 None libffi conda/linux-64::libffi-3.4.2-h7f98852_5 None libgcc-ng conda/linux-64::libgcc-ng-12.2.0-h65d4601_19 None libgomp conda/linux-64::libgomp-12.2.0-h65d4601_19 None libiconv conda/linux-64::libiconv-1.17-h166bdaf_0 None libmamba conda/linux-64::libmamba-1.0.0-h9eff5f0_2 None libmambapy conda/linux-64::libmambapy-1.0.0-py310hd5a56e8_2 None libnghttp2 conda/linux-64::libnghttp2-1.47.0-hff17c54_1 None libnsl conda/linux-64::libnsl-2.0.0-h5eee18b_0 None libsolv conda/linux-64::libsolv-0.7.22-h6239696_0 None libsqlite conda/linux-64::libsqlite-3.39.4-h753d276_0 None libssh2 conda/linux-64::libssh2-1.10.0-hf14f497_3 None libstdcxx-ng conda/linux-64::libstdcxx-ng-12.2.0-h46fd767_19 None libuuid conda/linux-64::libuuid-2.32.1-h7f98852_1000 None libxml2 conda/linux-64::libxml2-2.10.3-h7463322_0 None libzlib conda/linux-64::libzlib-1.2.13-h166bdaf_4 None lz4-c conda/linux-64::lz4-c-1.9.3-h9c3ff4c_1 None lzo conda/linux-64::lzo-2.10-h516909a_1000 None mamba conda/linux-64::mamba-1.0.0-py310h51d5547_2 None ncurses conda/linux-64::ncurses-6.3-h5eee18b_3 None openssl conda/linux-64::openssl-3.0.7-h166bdaf_0 None pip conda/noarch::pip-22.3.1-pyhd8ed1ab_0 None pybind11-abi conda/noarch::pybind11-abi-4-hd8ed1ab_3 None pycosat conda/linux-64::pycosat-0.6.4-py310h5764c6d_1 None pycparser conda/noarch::pycparser-2.21-pyhd3eb1b0_0 None pyopenssl conda/noarch::pyopenssl-22.1.0-pyhd8ed1ab_0 None pysocks conda/noarch::pysocks-1.7.1-pyha2e5f31_6 None python conda/linux-64::python-3.10.6-ha86cf86_0_cpython None python_abi conda/linux-64::python_abi-3.10-2_cp310 None readline conda/linux-64::readline-8.2-h5eee18b_0 None reproc conda/linux-64::reproc-14.2.4-h295c915_1 None reproc-cpp conda/linux-64::reproc-cpp-14.2.4-h295c915_1 None requests conda/noarch::requests-2.28.1-pyhd8ed1ab_1 None ruamel_yaml conda/linux-64::ruamel_yaml-0.15.100-py310h7f8727e_0 None setuptools conda/noarch::setuptools-65.5.1-pyhd8ed1ab_0 None tk conda/linux-64::tk-8.6.12-h27826a3_0 None toolz conda/linux-64::toolz-0.12.0-py310h06a4308_0 None tqdm conda/linux-64::tqdm-4.64.1-py310h06a4308_0 None tzdata conda/noarch::tzdata-2022f-h04d1e81_0 None urllib3 conda/linux-64::urllib3-1.26.12-py310h06a4308_0 None wheel conda/noarch::wheel-0.38.4-pyhd8ed1ab_0 None xz conda/linux-64::xz-5.2.6-h5eee18b_0 None yaml conda/linux-64::yaml-0.2.5-h7f98852_2 None yaml-cpp conda/linux-64::yaml-cpp-0.7.0-h27087fc_2 None zstd conda/linux-64::zstd-1.5.2-h6239696_4 None Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate mamba_1.0 # # To deactivate an active environment, use # # $ conda deactivate Retrieving notices: ...working... done DEBUG conda.gateways.logging:set_verbosity(236): verbosity set to 2 info libmamba Reading cache files '/tmp/tmpx5sjd583.*' for repo index 'installed' info libmamba Searching index cache file for repo 'https://:@/artifactory/api/conda/conda/linux-64/repodata.json' info libmamba Found cache at '/home//scratch/.conda/pkgs/cache/d4da7e53.json' info libmamba No valid cache found info libmamba Expired cache (or invalid mod/etag headers) found at '/home//scratch/.conda/pkgs' info libmamba Searching index cache file for repo 'https://:@/artifactory/api/conda/conda/noarch/repodata.json' info libmamba Found cache at '/home//scratch/.conda/pkgs/cache/c607b521.json' info libmamba No valid cache found info libmamba Expired cache (or invalid mod/etag headers) found at '/home//scratch/.conda/pkgs' info libmamba Starting to download targets info libcurl * Uses proxy env variable no_proxy == '' info libcurl * Uses proxy env variable https_proxy == 'http://:@' info libcurl * Couldn't find host in the (nil) file; using defaults info libcurl * Uses proxy env variable no_proxy == '' info libcurl * Uses proxy env variable https_proxy == 'http://:@' info libcurl * Couldn't find host in the (nil) file; using defaults info libcurl * Found bundle for host: 0x5649b23fb680 [serially] info libcurl * Trying ... info libcurl * Hostname '' was found in DNS cache info libcurl * Trying ... info libcurl * Connected to (nil) () port (#1) info libcurl * allocate connect buffer info libcurl * Establish HTTP proxy tunnel to :443 info libcurl * Proxy auth using Basic with user '' info libcurl * Server auth using Basic with user '' info libcurl > CONNECT :443 HTTP/1.1 Host: :443 Proxy-Authorization: Basic Proxy-Connection: Keep-Alive info libcurl < HTTP/1.1 200 Connection established info libcurl < info libcurl * Proxy replied 200 to CONNECT request info libcurl * CONNECT phase completed info libcurl * ALPN: offers http/1.1 info libcurl * CAfile: /etc/pki/tls/certs/ca-bundle.crt info libcurl * CApath: none info libcurl * TLSv1.0 (OUT), TLS header, Certificate Status (22): info libcurl * TLSv1.3 (OUT), TLS handshake, Client hello (1): info libcurl * Connected to (nil) () port (#0) info libcurl * allocate connect buffer info libcurl * Establish HTTP proxy tunnel to :443 info libcurl * Proxy auth using Basic with user '' info libcurl * Server auth using Basic with user '' info libcurl > CONNECT :443 HTTP/1.1 Host: :443 Proxy-Authorization: Basic Proxy-Connection: Keep-Alive info libcurl < HTTP/1.1 200 Connection established info libcurl < info libcurl * Proxy replied 200 to CONNECT request info libcurl * CONNECT phase completed info libcurl * ALPN: offers http/1.1 info libcurl * CAfile: /etc/pki/tls/certs/ca-bundle.crt info libcurl * CApath: none info libcurl * TLSv1.0 (OUT), TLS header, Certificate Status (22): info libcurl * TLSv1.3 (OUT), TLS handshake, Client hello (1): info libcurl * SSL connection timeout info libcurl * Closing connection 0 info libcurl * SSL connection timeout info libcurl * Closing connection 1 info libmamba Download error (28) Timeout was reached [https://:@/artifactory/api/conda/conda/linux-64/repodata.json] SSL connection timeout info libmamba Transfer done for 'conda/linux-64' info libmamba Transfer finalized, status: 0 [https://:@/artifactory/api/conda/conda/linux-64/repodata.json] 0 bytes info libmamba Unable to retrieve repodata (response: 0) for 'https://:@/artifactory/api/conda/conda/linux-64/repodata.json' info libmamba Download error (28) Timeout was reached [https://:@/artifactory/api/conda/conda/noarch/repodata.json] SSL connection timeout Download error (28) Timeout was reached [https://:@/artifactory/api/conda/conda/noarch/repodata.json] SSL connection timeout DEBUG conda.gateways.logging:set_verbosity(236): verbosity set to 2 # >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<< Traceback (most recent call last): File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/conda/exceptions.py", line 1129, in __call__ return func(*args, **kwargs) File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/mamba/mamba.py", line 935, in exception_converter raise e File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/mamba/mamba.py", line 928, in exception_converter exit_code = _wrapped_main(*args, **kwargs) File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/mamba/mamba.py", line 886, in _wrapped_main result = do_call(parsed_args, p) File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/mamba/mamba.py", line 754, in do_call exit_code = create(args, parser) File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/mamba/mamba.py", line 630, in create return install(args, parser, "create") File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/mamba/mamba.py", line 497, in install index = load_channels(pool, channels, repos) File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/mamba/utils.py", line 129, in load_channels index = get_index( File "/home//scratch/.conda/envs/mamba/lib/python3.10/site-packages/mamba/utils.py", line 110, in get_index is_downloaded = dlist.download(api.MAMBA_DOWNLOAD_FAILFAST) RuntimeError: Download error (28) Timeout was reached [https://:@/artifactory/api/conda/conda/noarch/repodata.json] SSL connection timeout `$ /home//scratch/.conda/envs/mamba/bin/mamba create --yes --name mamba_install_1.0 python=3.9 numpy` environment variables: CIO_TEST= CMAKE_PREFIX_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s CONDA_DEFAULT_ENV=mamba_1.0 CONDA_EXE=/home//scratch/.conda/envs/mamba/bin/conda CONDA_PREFIX=/home//scratch/.conda/envs/mamba_1.0 CONDA_PREFIX_1=/home//scratch/.conda/envs/mamba CONDA_PREFIX_2=/home//scratch/.conda/envs/mamba_0.24 CONDA_PREFIX_3=/home//scratch/.conda/envs/mamba_0.23 CONDA_PREFIX_4=/home//scratch/.conda/envs/mamba_0.22 CONDA_PREFIX_5=/home//scratch/.conda/envs/mamba_0.21 CONDA_PREFIX_6=/home//scratch/.conda/envs/mamba_0.20 CONDA_PREFIX_7=/home//scratch/.conda/envs/mamba_0.19 CONDA_PROMPT_MODIFIER=(mamba_1.0) CONDA_PYTHON_EXE=/home//scratch/.conda/envs/mamba/bin/python CONDA_ROOT=/home//scratch/.conda/envs/mamba CONDA_SHLVL=8 CONDA_VERBOSITY=2 CPATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/include/python3.8 CPLUS_INCLUDE_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/include CURL_CA_BUNDLE= C_INCLUDE_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/include FPATH=/softs/lmod/tools/fpath FTP_PROXY=\ GIT_EXEC_PATH=/softs/projets/ul/git/2.34.1/bin HTTPS_PROXY=\ HTTP_PROXY=\ LD_LIBRARY_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/lib LIBRARY_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/lib LMOD_DEFAULT_MODULEPATH=/softs/modulefiles/:/softs/projets/modulefiles/ LMOD_PACKAGE_PATH=/softs/lmod/install/lmod_site/ MANPATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/share/man:/softs/rh7/spack_install/li nux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/man:/opt/pbs/man:: MODULEPATH=/softs/modulefiles:/softs/projets/modulefiles:/softs/rh7/spack_modulef iles/linux-centos7-x86_64 NO_PROXY=\ PATH=/home//scratch/.conda/envs/mamba_1.0/bin:/home//scra tch/.conda/envs/mamba/condabin:/softs/projets/ul/jfrog-cli/2.12.1:/sof ts/projets/ul/git/2.34.1/bin:/softs/rh7/spack_install/linux-centos7- x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/bin:/usr/lib64/qt- 3.3/bin:/opt/pbs/bin:/opt/pbs/tools/bin:/usr/lpp/mmfs/bin:/usr/local/s bin:/usr/local/bin:/usr/bin:/usr/sbin:/opt/ibutils/bin:/home/ /bin:/home//.local/bin:/home//bin PKG_CONFIG_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/lib/pkgconfig PYTHON_HOME=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s REQUESTS_CA_BUNDLE=/etc/pki/tls/cert.pem SSL_CERT_FILE= __LMOD_REF_COUNT_CMAKE_PREFIX_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s:1 __LMOD_REF_COUNT_CPATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/include/python3.8:1 __LMOD_REF_COUNT_CPLUS_INCLUDE_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/include:1 __LMOD_REF_COUNT_C_INCLUDE_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/include:1 __LMOD_REF_COUNT_LD_LIBRARY_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/lib:1 __LMOD_REF_COUNT_LIBRARY_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/lib:1 __LMOD_REF_COUNT_MANPATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/share/man:1;/softs/rh7/spack_install/ linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/man:1;/opt/pbs/man:1 __LMOD_REF_COUNT_MODULEPATH=/softs/modulefiles:1;/softs/projets/modulefiles:1;/softs/rh7/spack_mod ulefiles/linux-centos7-x86_64:1 __LMOD_REF_COUNT_PATH=/softs/projets/ul/jfrog-cli/2.12.1:1;/softs/projets/ul/git/2.34.1/bin: 1;/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python- 3.8.4-quigmdjvagbs5t4qqlb7yhmgugpa4q5s/bin:1;/usr/lib64/qt- 3.3/bin:1;/opt/pbs/bin:1;/opt/pbs/tools/bin:1;/usr/lpp/mmfs/bin:1;/usr /local/sbin:2;/usr/local/bin:1;/usr/bin:1;/usr/sbin:1;/opt/ibutils/bin :1 __LMOD_REF_COUNT_PKG_CONFIG_PATH=/softs/rh7/spack_install/linux-centos7-x86_64/gcc-10.2.0/python-3.8.4- quigmdjvagbs5t4qqlb7yhmgugpa4q5s/lib/pkgconfig:1 ftp_proxy=\ http_proxy=\ https_proxy=\ no_proxy=\ active environment : mamba_1.0 active env location : /home//scratch/.conda/envs/mamba_1.0 shell level : 8 user config file : /home//.condarc populated config files : /home//.condarc conda version : 22.9.0 conda-build version : not installed python version : 3.10.6.final.0 virtual packages : __cuda=11.4=0 __linux=3.10.0=0 __glibc=2.17=0 __unix=0=0 __archspec=1=x86_64 base environment : /home//scratch/.conda/envs/mamba (writable) conda av data dir : /home//scratch/.conda/envs/mamba/etc/conda conda av metadata url : None channel URLs : https://:@/artifactory/api/conda/conda/linux-64 https://:@/artifactory/api/conda/conda/noarch package cache : /home//scratch/.conda/pkgs envs directories : /home//scratch/.conda/envs /home//scratch/.conda/envs/mamba/envs /home//.conda/envs platform : linux-64 user-agent : conda/22.9.0 requests/2.28.1 CPython/3.10.6 Linux/3.10.0-1160.el7.x86_64 centos/7.9.2009 glibc/2.17 UID:GID : 120961:8305 netrc file : None offline mode : False An unexpected error has occurred. Conda has prepared the above report. Looking for: ['python=3.9', 'numpy'] info libmamba Freeing pool. (mamba_1.0) ```

curl version :

$curl --version
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.44 zlib/1.2.7 libidn/1.28 libssh2/1.8.0
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets
(mamba_1.0)
jonashaag commented 1 year ago

Thanks for the detailed bug report! (Formatting is broken in the expanded section.)

What's the exact value of your NO_PROXY variable? Can you try to download the repodata.json URL with the curl command line, does it work?

usinelogicielle commented 1 year ago

I try with curl and the same configuration proxy and it works

export NO_PROXY=domains.com

I also try NO_PROXY=*.domains.com

jonashaag commented 1 year ago

Can you please show your condarc?

In the output above it shows that the proxy environment variables are empty. Are they actually empty or is it displayed incorrectly?

usinelogicielle commented 1 year ago
ssl_verify: /etc/pki/tls/certs/ca-bundle.crt
channel_alias: https://<login>:<password>@<artifactory.domains.com>/artifactory/api/conda/
default_channels:
   - conda
channels:
  - defaults
# Permet de ne pas perdre de place sur son HOME
envs_dirs:
  - ~/scratch/.conda/envs
pkgs_dirs:
  - ~/scratch/.conda/pkgs

There isn't proxy configuration on condarc. only HTTPS_PROXY, HTTP_PROXY and NO_PROXY variable environment. At the end, the installation will be in Dockerfile and we dont' want stock .condarc organization information.

jonashaag commented 1 year ago

Is this a mistake in the debug logs above?

environment variables:
CIO_TEST=
...
GIT_EXEC_PATH=/softs/projets/ul/git/2.34.1/bin
HTTPS_PROXY=
HTTP_PROXY=

What's the curl command that works? Can you please post curl -v ... for the repodata file? I'm trying to compare what a vanilla curl command does vs. what Mamba does.

usinelogicielle commented 1 year ago

It's an github markdown error. In the real log, there is HTTP_PROXY=\<set>

The curl with -v

$curl -v  https://<login>:<password>@<artifactory.domains.com>/artifactory/api/conda/conda/noarch/repodata.json -o repo.json
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to <artifactory.domains.com> port 443 (#0)
*   Trying <artifactory_ip>...
* Connected to <artifactory.domains.com> (<artifactory_ip>) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=<artifactory.domains.com>,OU=SSL Client Serveur,OU=0002 775665912,OU=CST,O=<ORGANISATION>,C=FR
*       start date: Aug 29 15:10:23 2022 GMT
*       expire date: Sep 22 15:10:23 2023 GMT
*       common name: <artifactory.domains.com>
*       issuer: CN=<CA_INTERMEDIATE>,OU=0002 775665912,O=<ORGANISATION>,C=FR
* Server auth using Basic with user '<login>'
> GET /artifactory/api/conda/conda/noarch/repodata.json HTTP/1.1
> Authorization: Basic <basic_auth>
> User-Agent: curl/7.29.0
> Host: <artifactory.domains.com>
> Accept: */*
>
< HTTP/1.1 200
< Date: Thu, 10 Nov 2022 15:16:27 GMT
< Content-Type: application/json
< Content-Length: 80722901
< Connection: keep-alive
< X-JFrog-Version: Artifactory/7.33.12 73312900
< X-Artifactory-Id: 3d138bb0dfcf9bd297d9cfc3214d6fd624fc4797
< X-Artifactory-Node-Id: <artifactory_name>.<ORGANISATION>.fr
< Last-Modified: Thu, 10 Nov 2022 15:08:46 GMT
< ETag: e3c29d24fa1215268f8dcb39b7c6719db0fc1d62
< X-Checksum-Sha1: e3c29d24fa1215268f8dcb39b7c6719db0fc1d62
< X-Checksum-Sha256: 4cd8624e0767199a0e27e18f5abbf06a22e7cb7085c39b67935730a3b457da42
< X-Checksum-Md5: 774451a2d52673fb4987ecba0743e566
< Accept-Ranges: bytes
< X-Artifactory-Filename: repodata.json
< Content-Disposition: attachment; filename="repodata.json"
< Cache-Control: no-store
< Set-Cookie: BIGipServer<artifactory.domains.com>_pool=4046092298.47873.0000; path=/; Httponly; Secure
<
jonashaag commented 1 year ago

Did you actually pass the HTTP* environment variables to this curl command? Because if I do that it will print some proxy stuff:

http_proxy=localhost:123 curl -v http://google.com
* Uses proxy env variable http_proxy == 'localhost:123'
...

Interestingly it doesn't seem to care about a HTTP_PROXY variable, only lowercase.

jonashaag commented 1 year ago
no_proxy=google.com http_proxy=localhost:123 curl -v http://google.com
* Uses proxy env variable no_proxy == 'google.com'
...
jonashaag commented 1 year ago

Can you please fix the formatting in the original post?

usinelogicielle commented 1 year ago

I fix the formatting I retry the curl without success for more log.

jonashaag commented 1 year ago

I don’t know what the last message means.

usinelogicielle commented 1 year ago

Hi @jonashaag I retry the curl command in my environment with more verbosity. But I don't have the same logs with proxy information.

jonashaag commented 1 year ago

OK, it would be very helpful if you could come up with a curl command that uses the no_proxy and http(s)_proxy environment variables and succeeds at downloading the file.

usinelogicielle commented 1 year ago

Here the result :

http_proxy="http://<proxy_login>:<proxy_pwd>@<proxy_url>" https_proxy="http://<proxy_login>:<proxy_pwd>@<proxy_url>"  no_proxy=<domain_name> curl -vv  https://<artifactory_login>:<artifactory_pwd>@<artifactory_url>/artifactory/api/conda/conda/noarch/repodata.json -o repo.json
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to <artifactory_url> port 443 (#0)
*   Trying <artifactory_ip>...
* Connected to <artifactory_url> (<artifactory_ip>) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=<artifactory_url>,OU=SSL Client Serveur,OU=0002 775665912,OU=CST,O=<ORGANISATION>,C=FR
*       start date: Aug 29 15:10:23 2022 GMT
*       expire date: Sep 22 15:10:23 2023 GMT
*       common name: <artifactory_url>
*       issuer: CN=AC TECHNIQUE SERVEURS 2028,OU=0002 775665912,O=<ORGANISATION>,C=FR
* Server auth using Basic with user '<proxy_login>'
> GET /artifactory/api/conda/conda/noarch/repodata.json HTTP/1.1
> Authorization: Basic <curl_basic_auth>
> User-Agent: curl/7.29.0
> Host: <artifactory_url>
> Accept: */*
>
< HTTP/1.1 200
< Date: Mon, 14 Nov 2022 16:19:25 GMT
< Content-Type: application/json
< Content-Length: 80946280
< Connection: keep-alive
< X-JFrog-Version: Artifactory/7.33.12 73312900
< X-Artifactory-Id: acfa4c7ed93b8ddf94db898c8d1f19965ec5315f
< X-Artifactory-Node-Id: <artifactory_machine_name>
< Last-Modified: Mon, 14 Nov 2022 15:08:38 GMT
< ETag: d35ddc3c3c73248a25e2ffb87424241a5d6cf89a
< X-Checksum-Sha1: d35ddc3c3c73248a25e2ffb87424241a5d6cf89a
< X-Checksum-Sha256: 60650aa1aba66193c2bbd4654f981b11e9ba5e5f677f32eedc585aaf25aaa5ef
< X-Checksum-Md5: 0bfa3c83a4db706cc8197568da8e9aa2
< Accept-Ranges: bytes
< X-Artifactory-Filename: repodata.json
< Content-Disposition: attachment; filename="repodata.json"
< Cache-Control: no-store
< Set-Cookie: BIGipServer<artifactory_url>_pool=4062869514.47873.0000; path=/; Httponly; Secure
<
{ [data not shown]
100 77.1M  100 77.1M    0     0  98.1M      0 --:--:-- --:--:-- --:--:-- 98.2M
* Connection #0 to host <artifactory_url> left intact
jonashaag commented 1 year ago

Hm, not sure what happens here -- does curl use the environment variables but not print? Because I can't find any reference to proxy environment variables in the output

jonashaag commented 1 year ago

Is <domain_name> a superdomain of <artifactory_url>? Ie. domain_name=blah.com, artifactory_url=something.blah.com

usinelogicielle commented 1 year ago

Hello @jonashaag ,

Good news, I reproduced the problem with recent curl version. I'm using singularity, alternative to docker, to use recent version of curl. I have the proxy informations and I reproduce the problem.

And yes <domain_name> is the same domain that artifactory and the proxy urls.

$singularity exec ~/scratch/singularity_image/curl_latest.sif sh -c  'http_proxy="http://<proxy_login>:<proxy_pwd>@<proxy_url>" https_proxy="http://<proxy_login>:<proxy_pwd>@<proxy_url>"  no_proxy=<artifactory_domains> curl -vv  https://<artifactory_login>:<artifactory_url>@<artifactory_url>/artifactory/api/conda/conda/noarch/repodata.json -o repo.json'

* Uses proxy env variable no_proxy == '<artifactory_domains>'
* Uses proxy env variable https_proxy == 'http://<proxy_login>:<proxy_pwd>@<proxy_url>'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying <proxy_ip>...
* Connected to (nil) (<proxy_ip>) port <proxy_port> (#0)
* allocate connect buffer
* Establish HTTP proxy tunnel to <artifactory_url>:443
* Proxy auth using Basic with user '<proxy_login>'
* Server auth using Basic with user '<proxy_login>'
> CONNECT <artifactory_url>:443 HTTP/1.1
> Host: <artifactory_url>:443
> Proxy-Authorization: Basic <proxy_auth>
> User-Agent: curl/7.86.0-DEV
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed
* ALPN: offers h2
* ALPN: offers http/1.1
*  CAfile: /cacert.pem
*  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
  0     0    0     0    0     0      0      0 --:--:--  0:00:16 --:--:--     0* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to <artifactory_url>:443
  0     0    0     0    0     0      0      0 --:--:--  0:00:17 --:--:--     0
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to <artifactory_url>:443

I have new question. Is mamba using that own curl library? Or is it using the curl OS version?

jonashaag commented 1 year ago

You can see the curl version of Mamba like so:

mamba list -n base libcurl

And for Micromamba:

micromamba info | grep curl
usinelogicielle commented 1 year ago

@jonashaag I think, i found the problem. There is a regression in curl in 7.86.0 version.

This is the issue : https://github.com/curl/curl/issues/9884 This is corrected in master branch but there isn't new version for now.

Do you return to use libcurl 7.85.0 or wait the new version?

jonashaag commented 1 year ago

Nice find! You should be able to upgrade curl when it’s released.

usinelogicielle commented 1 year ago

We can closed this issue. I tested with mamba 1.3.1 and libcurl 7.87.0 and it works now. Thanks for the help.