search

mamba-org / mamba

The Fast Cross-Platform Package Manager
https://mamba.readthedocs.io
BSD 3-Clause "New" or "Revised" License
6.97k stars 359 forks source link

md5 and sha256 sum unknown #2674

Open SinisterMJ opened 1 year ago

SinisterMJ commented 1 year ago

Troubleshooting docs

How did you install Mamba?

Mambaforge

Search tried in issue tracker

Found nothing similar.

Latest version of Mamba

Tried in Conda?

I do not have this problem with Conda, just with Mamba

Describe your issue

In our dockerfile we have following line

RUN conda install mamba -q -c conda-forge \ && mamba env update -q -n base -f environment.yml

This runs fine with mamba 1.4.7 (checked by pinning version to mamba==1.4.7), but with 1.4.9 it crashes:

The following NEW packages will be INSTALLED:
  libzlib            conda-forge/linux-64::libzlib-1.2.13-hd590300_5 
  llvm-openmp        conda-forge/linux-64::llvm-openmp-16.0.6-h4dfa4b3_0 
  mamba              conda-forge/linux-64::mamba-1.4.9-py311h3072747_0 
  python_abi         conda-forge/linux-64::python_abi-3.11-2_cp311 
The following packages will be REMOVED:
  libgomp-11.2.0-h1234567_1
The following packages will be UPDATED:
  libcurl               pkgs/main::libcurl-8.1.1-h251f7ec_1 --> conda-forge::libcurl-8.1.2-h409715c_0 
  libgcc-ng          pkgs/main::libgcc-ng-11.2.0-h1234567_1 --> conda-forge::libgcc-ng-13.1.0-he5830b7_0 
  libmamba             pkgs/main::libmamba-1.4.1-h2dafd23_1 --> conda-forge::libmamba-1.4.9-h658169a_0 
  libmambapy         pkgs/main::libmambapy-1.4.1-py311h2da~ --> conda-forge::libmambapy-1.4.9-py311h527f279_0 
  libsolv              pkgs/main::libsolv-0.7.22-he621ea3_0 --> conda-forge::libsolv-0.7.24-hfc55251_1 
  libstdcxx-ng       pkgs/main::libstdcxx-ng-11.2.0-h12345~ --> conda-forge::libstdcxx-ng-13.1.0-hfd8a6a1_0 
  openssl               pkgs/main::openssl-3.0.9-h7f8727e_0 --> conda-forge::openssl-3.1.1-hd590300_1 
  zlib                    pkgs/main::zlib-1.2.13-h5eee18b_0 --> conda-forge::zlib-1.2.13-hd590300_5 
The following packages will be SUPERSEDED by a higher-priority channel:
  _libgcc_mutex           pkgs/main::_libgcc_mutex-0.1-main --> conda-forge::_libgcc_mutex-0.1-conda_forge 
  _openmp_mutex          pkgs/main::_openmp_mutex-5.1-1_gnu --> conda-forge::_openmp_mutex-4.5-2_kmp_llvm 
  ca-certificates    pkgs/main::ca-certificates-2023.05.30~ --> conda-forge::ca-certificates-2023.5.7-hbcca054_0 
  certifi            pkgs/main/linux-64::certifi-2023.5.7-~ --> conda-forge/noarch::certifi-2023.5.7-pyhd8ed1ab_0 
Proceed ([y]/n)? 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
warning  libmamba Could not validate package '/opt/miniconda/pkgs/six-1.16.0-pyhd3eb1b0_1/info/repodata_record.json': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Extracted package cache '/opt/miniconda/pkgs/six-1.16.0-pyhd3eb1b0_1 couldn't be validated due to runtime error: Could not set URL (code: 14 - url = /six-1.16.0-pyhd3eb1b0_1.conda)
warning  libmamba Could not validate package '/opt/miniconda/pkgs/six-1.16.0-pyhd3eb1b0_1.conda': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Could not validate package '/opt/miniconda/pkgs/pycparser-2.21-pyhd3eb1b0_0/info/repodata_record.json': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Extracted package cache '/opt/miniconda/pkgs/pycparser-2.21-pyhd3eb1b0_0 couldn't be validated due to runtime error: Could not set URL (code: 14 - url = /pycparser-2.21-pyhd3eb1b0_0.conda)
warning  libmamba Could not validate package '/opt/miniconda/pkgs/pycparser-2.21-pyhd3eb1b0_0.conda': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Could not validate package '/opt/miniconda/pkgs/jsonpointer-2.1-pyhd3eb1b0_0/info/repodata_record.json': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Extracted package cache '/opt/miniconda/pkgs/jsonpointer-2.1-pyhd3eb1b0_0 couldn't be validated due to runtime error: Could not set URL (code: 14 - url = /jsonpointer-2.1-pyhd3eb1b0_0.conda)
warning  libmamba Could not validate package '/opt/miniconda/pkgs/jsonpointer-2.1-pyhd3eb1b0_0.conda': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Could not validate package '/opt/miniconda/pkgs/charset-normalizer-2.0.4-pyhd3eb1b0_0/info/repodata_record.json': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Extracted package cache '/opt/miniconda/pkgs/charset-normalizer-2.0.4-pyhd3eb1b0_0 couldn't be validated due to runtime error: Could not set URL (code: 14 - url = /charset-normalizer-2.0.4-pyhd3eb1b0_0.conda)
warning  libmamba Could not validate package '/opt/miniconda/pkgs/charset-normalizer-2.0.4-pyhd3eb1b0_0.conda': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Could not validate package '/opt/miniconda/pkgs/jsonpatch-1.32-pyhd3eb1b0_0/info/repodata_record.json': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Extracted package cache '/opt/miniconda/pkgs/jsonpatch-1.32-pyhd3eb1b0_0 couldn't be validated due to runtime error: Could not set URL (code: 14 - url = /jsonpatch-1.32-pyhd3eb1b0_0.conda)
warning  libmamba Could not validate package '/opt/miniconda/pkgs/jsonpatch-1.32-pyhd3eb1b0_0.conda': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Could not validate package '/opt/miniconda/pkgs/certifi-2023.5.7-pyhd8ed1ab_0/info/repodata_record.json': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
warning  libmamba Extracted package cache '/opt/miniconda/pkgs/certifi-2023.5.7-pyhd8ed1ab_0 couldn't be validated due to runtime error: Could not set URL (code: 14 - url = /certifi-2023.5.7-pyhd8ed1ab_0.conda)
warning  libmamba Could not validate package '/opt/miniconda/pkgs/certifi-2023.5.7-pyhd8ed1ab_0.conda': md5 and sha256 sum unknown.
    Set safety_checks to disabled to override this warning.
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
    Traceback (most recent call last):
      File "/opt/miniconda/lib/python3.11/site-packages/conda/exception_handler.py", line 16, in __call__
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/opt/miniconda/lib/python3.11/site-packages/conda_env/cli/main.py", line 49, in do_call
        exit_code = getattr(module, func_name)(args, parser)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/miniconda/lib/python3.11/site-packages/conda/notices/core.py", line 123, in wrapper
        return func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
      File "/opt/miniconda/lib/python3.11/site-packages/conda_env/cli/main_update.py", line 139, in execute
        result[installer_type] = installer.install(prefix, specs, args, env)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/miniconda/lib/python3.11/site-packages/mamba/mamba_env.py", line 169, in mamba_install
        conda_transaction = to_txn(
                            ^^^^^^^
      File "/opt/miniconda/lib/python3.11/site-packages/mamba/utils.py", line 464, in to_txn
        final_precs = compute_final_precs(
                      ^^^^^^^^^^^^^^^^^^^^
      File "/opt/miniconda/lib/python3.11/site-packages/mamba/utils.py", line 410, in compute_final_precs
        raise ValueError("missing key {} in channels: {}".format(key, lookup_dict))
    ValueError: missing key https://repo.anaconda.com/pkgs/main/noarch/six-1.16.0-pyhd3eb1b0_1.conda in channels: {'https://repo.anaconda.com/pkgs/main/linux-64': {'platform': 'linux-64', 'url': 'https://repo.anaconda.com/pkgs/main/linux-64', 'channel': pkgs/main[linux-64,noarch], 'needs_finalising': True}, 'https://repo.anaconda.com/pkgs/main/noarch': {'platform': 'noarch', 'url': 'https://repo.anaconda.com/pkgs/main/noarch', 'channel': pkgs/main[linux-64,noarch], 'needs_finalising': True}, 'https://repo.anaconda.com/pkgs/r/linux-64': {'platform': 'linux-64', 'url': 'https://repo.anaconda.com/pkgs/r/linux-64', 'channel': pkgs/r[linux-64,noarch], 'needs_finalising': True}, 'https://repo.anaconda.com/pkgs/r/noarch': {'platform': 'noarch', 'url': 'https://repo.anaconda.com/pkgs/r/noarch', 'channel': pkgs/r[linux-64,noarch], 'needs_finalising': True}, 'https://conda.anaconda.org/conda-forge/linux-64': {'platform': 'linux-64', 'url': 'https://conda.anaconda.org/conda-forge/linux-64', 'channel': conda-forge[linux-64,noarch], 'needs_finalising': False}, 'https://conda.anaconda.org/conda-forge/noarch': {'platform': 'noarch', 'url': 'https://conda.anaconda.org/conda-forge/noarch', 'channel': conda-forge[linux-64,noarch], 'needs_finalising': False}, 'https://conda.anaconda.org/pytorch/linux-64': {'platform': 'linux-64', 'url': 'https://conda.anaconda.org/pytorch/linux-64', 'channel': pytorch[linux-64,noarch], 'needs_finalising': True}, 'https://conda.anaconda.org/pytorch/noarch': {'platform': 'noarch', 'url': 'https://conda.anaconda.org/pytorch/noarch', 'channel': pytorch[linux-64,noarch], 'needs_finalising': True}}
`$ /opt/miniconda/condabin/mamba update -q -n base -f environment.yml`
  environment variables:
                 CIO_TEST=<not set>
  CONDA_AUTO_UPDATE_CONDA=false
                CONDA_DIR=/opt/miniconda
                CONDA_EXE=/opt/miniconda/bin/conda
         CONDA_PYTHON_EXE=/opt/miniconda/bin/python
               CONDA_ROOT=/opt/miniconda
              CONDA_SHLVL=0
           CURL_CA_BUNDLE=<not set>
          LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
               LD_PRELOAD=<not set>
             LIBRARY_PATH=/usr/local/cuda/lib64/stubs
                     PATH=/opt/miniconda/condabin:/home/uxavuc_user/bin:/opt/miniconda/bin:/home
                          /uxavuc_user/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/
                          sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>
     active environment : None
            shell level : 0
       user config file : /home/uxavuc_user/.condarc
 populated config files : 
          conda version : 23.5.2
    conda-build version : not installed
         python version : 3.11.4.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=5.15.0=0
                          __unix=0=0
       base environment : /opt/miniconda  (writable)
      conda av data dir : /opt/miniconda/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/miniconda/pkgs
                          /home/uxavuc_user/.conda/pkgs
       envs directories : /opt/miniconda/envs
                          /home/uxavuc_user/.conda/envs
               platform : linux-64
             user-agent : conda/23.5.2 requests/2.29.0 CPython/3.11.4 Linux/5.15.0-76-generic ubuntu/22.04.2 glibc/2.35
                UID:GID : 1009:1009
             netrc file : None
           offline mode : False
An unexpected error has occurred. Conda has prepared the above report.
error building image: error building stage: failed to execute command: waiting for process to exit: exit status 1

mamba info / micromamba info

mamba version : 1.4.9
     active environment : None
            shell level : 0
       user config file : /home/uxavuc_user/.condarc
 populated config files : 
          conda version : 23.5.2
    conda-build version : not installed
         python version : 3.11.4.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=5.15.0=0
                          __unix=0=0
       base environment : /opt/miniconda  (writable)
      conda av data dir : /opt/miniconda/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/miniconda/pkgs
                          /home/uxavuc_user/.conda/pkgs
       envs directories : /opt/miniconda/envs
                          /home/uxavuc_user/.conda/envs
               platform : linux-64
             user-agent : conda/23.5.2 requests/2.29.0 CPython/3.11.4 Linux/5.15.0-76-generic ubuntu/22.04.2 glibc/2.35
                UID:GID : 1009:1009
             netrc file : None
           offline mode : False

Logs

No response

environment.yml

name: uxavuc
channels:
  - defaults
  - conda-forge
  - pytorch
  # - apple # only necessary for apple silicon
dependencies:
  - _openmp_mutex=*=*_gnu
  - black
  - click # Necessary for experiment_cli
  - flake8
  - hypothesis # Better intergration tests
  - ipykernel
  - lime
  - loguru
  - matplotlib
  - mlflow # Tracking machine learning experiments
  - numpy
  - openpyxl
  - openssl
  - optuna
  - pandas
  - paramiko
  - pip
  - pre-commit # Pre commit hooks, used for black & flake8 checking
  - PyJWT
  - pytest
  - python-dotenv
  - python=3.10
  - pyyaml
  - scikit-learn
  - seaborn
  - shap
  - toolz
  - treelib
  - umap-learn
  - xlsxwriter
  # - tensorflow-deps # only necessary for apple silicon
  - pip:
      - kaleido
      - dvc-s3
      - dvc-ssh
      - dvc==2.34.0
      - packaging
      - pytest-watch # For automated re-execution of tests
      - tensorflow==2.9.0; sys_platform!="darwin" or platform_machine != 'arm64'
      - tensorflow-probability==0.17.0 # this release is specifically tailored to the TF 2.9.0 to shut down the warning

      # Apple Silicon Specific stuff (without Rosetta)
      - tensorflow-macos==2.9.0; sys_platform == 'darwin' and platform_machine == 'arm64'
      - tensorflow-metal==0.5; sys_platform == 'darwin' and platform_machine == 'arm64'

      # PyTorch
      - torchmetrics
      - torch==1.13.0
      - torchvision==0.14
      - torchaudio==0.13

      - kubernetes # To trigger CLI commands in deployed webapi
      - requests
      - urllib3 # Necessary to configure mlflow server communication
      - wfdb==3.4.0

      # Dependencies for dashy / reporting
      - bokeh==2.4.2
      - cachetools==5.0.0
      - dash_bootstrap_components
      - dash-bootstrap-components==1.1.0
      - dash==2.4.1
      - feather-format==0.4.1
      - feather-format==0.4.1
      - Flask-Caching
      - flask-caching==1.10.1
      - flask==2.1.2
      - holoviews
      - hvplot
      - jupyter-bokeh==3.0.3 # Keep this before holoviews
      - jupytext
      - plotly==5.8.0
      - pyarrow==8.0.0
      - types-cachetools==5.2.1
      - dash_defer_js_import
      # Dependencies for documentation
      - m2r2
      - nbsphinx
      - sphinx
      - sphinx-autodoc-typehints
      - sphinx-typlog-theme
      - sphinxcontrib-plantuml

      # Install Packages:
      - setuptools
      - setuptools-cpp

  # NOTE: The package must be installed from here, due to setup.py can only be installed
  # from https://pypi.org/project/nodejs/ and there is no version 14 in it yet.
  - nodejs>=16

  # NOTE: We enforce a version >1, since conda would otherwise prefer a rather old
  # version in the defaults channel.
  # NOTE: The package must be installed from here, due to setup.py can only be installed
  # from https://pypi.org/project/yarn/ and there is no version >1 in it yet.
  - yarn>1

~/.condarc

No response

JohanMabille commented 1 year ago

@SinisterMJ thanks for reporting. Can you post the list of the packages in you environment before trying to update? (you can simply copy paste the output of mamba list once in your environment).

SinisterMJ commented 1 year ago

The output of mamba list:

# packages in environment at /opt/miniconda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
boltons                   23.0.0          py311h06a4308_0  
brotlipy                  0.7.0           py311h5eee18b_1002  
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.19.0               h5eee18b_0  
ca-certificates           2023.5.7             hbcca054_0    conda-forge
certifi                   2023.5.7           pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py311h5eee18b_3  
charset-normalizer        2.0.4              pyhd3eb1b0_0  
conda                     23.5.2          py311h38be061_0    conda-forge
conda-content-trust       0.1.3           py311h06a4308_0  
conda-libmamba-solver     23.5.0          py311h06a4308_0  
conda-package-handling    2.1.0           py311h06a4308_0  
conda-package-streaming   0.8.0           py311h06a4308_0  
cryptography              39.0.1          py311h9ce1e76_2  
fmt                       9.1.0                hdb19cb5_0  
icu                       58.2                 he6710b0_3  
idna                      3.4             py311h06a4308_0  
jsonpatch                 1.32               pyhd3eb1b0_0  
jsonpointer               2.1                pyhd3eb1b0_0  
krb5                      1.20.1               h143b758_1  
ld_impl_linux-64          2.38                 h1181459_1  
libarchive                3.6.2                h6ac8c49_2  
libcurl                   8.1.2                h409715c_0    conda-forge
libedit                   3.1.20221030         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 13.1.0               he5830b7_0    conda-forge
libmamba                  1.4.9                h658169a_0    conda-forge
libmambapy                1.4.9           py311h527f279_0    conda-forge
libnghttp2                1.52.0               h2d74bed_1  
libsolv                   0.7.24               hfc55251_1    conda-forge
libssh2                   1.10.0               hdbd6064_2  
libstdcxx-ng              13.1.0               hfd8a6a1_0    conda-forge
libuuid                   1.41.5               h5eee18b_0  
libxml2                   2.10.3               hcbfbd50_0  
libzlib                   1.2.13               hd590300_5    conda-forge
llvm-openmp               16.0.6               h4dfa4b3_0    conda-forge
lz4-c                     1.9.4                h6a678d5_0  
mamba                     1.4.9           py311h3072747_0    conda-forge
ncurses                   6.4                  h6a678d5_0  
openssl                   3.1.1                hd590300_1    conda-forge
packaging                 23.0            py311h06a4308_0  
pcre2                     10.37                he7ceb23_1  
pip                       23.1.2          py311h06a4308_0  
pluggy                    1.0.0           py311h06a4308_1  
pybind11-abi              4                    hd3eb1b0_1  
pycosat                   0.6.4           py311h5eee18b_0  
pycparser                 2.21               pyhd3eb1b0_0  
pyopenssl                 23.0.0          py311h06a4308_0  
pysocks                   1.7.1           py311h06a4308_0  
python                    3.11.4               h955ad1f_0  
python_abi                3.11                    2_cp311    conda-forge
readline                  8.2                  h5eee18b_0  
reproc                    14.2.4               h295c915_1  
reproc-cpp                14.2.4               h295c915_1  
requests                  2.29.0          py311h06a4308_0  
ruamel.yaml               0.17.21         py311h5eee18b_0  
setuptools                67.8.0          py311h06a4308_0  
six                       1.16.0             pyhd3eb1b0_1  
sqlite                    3.41.2               h5eee18b_0  
tk                        8.6.12               h1ccaba5_0  
toolz                     0.12.0          py311h06a4308_0  
tqdm                      4.65.0          py311h92b7b1e_0  
tzdata                    2023c                h04d1e81_0  
urllib3                   1.26.16         py311h06a4308_0  
wheel                     0.38.4          py311h06a4308_0  
xz                        5.4.2                h5eee18b_0  
yaml-cpp                  0.7.0                h295c915_1  
zlib                      1.2.13               hd590300_5    conda-forge
zstandard                 0.19.0          py311h5eee18b_0  
zstd                      1.5.5                hc292b87_0
theavey commented 1 year ago

I have a similar issue, too. On Windows. Kinda guessing it's related to #2668 (because this started happening recently and they both talk about URLs). This is the stacktrace I get when doing a mamba update --file environment.yml

21:09:31       Traceback (most recent call last):
21:09:31         File "D:\ProgramData\Miniconda3\lib\site-packages\conda\exception_handler.py", line 16, in call
21:09:31           return func(args, **kwargs)
21:09:31         File "D:\ProgramData\Miniconda3\lib\site-packages\conda_env\cli\main.py", line 49, in do_call
21:09:31           exit_code = getattr(module, func_name)(args, parser)
21:09:31         File "D:\ProgramData\Miniconda3\lib\site-packages\conda\notices\core.py", line 123, in wrapper
21:09:31           return func(args, **kwargs)
21:09:31         File "D:\ProgramData\Miniconda3\lib\site-packages\conda_env\cli\main_update.py", line 139, in execute
21:09:31           result[installer_type] = installer.install(prefix, specs, args, env)
21:09:31         File "D:\ProgramData\Miniconda3\lib\site-packages\mamba\mamba_env.py", line 153, in mamba_install
21:09:31           transaction.fetch_extract_packages()
21:09:31       RuntimeError: Could not set URL (code: 14 - url = /python-3.9.16-h4de0772_0_cpython.conda)
21:09:31