search

conda / conda-package-handling

Create and extract conda packages of various formats
https://conda.github.io/conda-package-handling/
BSD 3-Clause "New" or "Revised" License
26 stars 37 forks source link

CPH 2.0.2 cannot extract .tar.bz2 that are actually uncompressed .tar, breaking a conda-build test #186

Closed kenodegard closed 1 year ago

kenodegard commented 1 year ago

Checklist

What happened?

The following tests fail with conda-package-handling 2.0.2 but work fine with 1.9.0:

conda_build/tests/test_index.py::test_file_index_noarch_osx64_1
conda_build/tests/test_index.py::test_index_of_removed_pkg

After some digging, it appears to be coming from bz2/tarfile. I can replicate these errors locally.

Xref https://github.com/conda/conda-build/actions/runs/3842350142/jobs/6543572627#step:8:3167

Conda Info

active environment : conda-build-test
    active env location : /Users/kodegard/.conda/22.11.1/3.10/envs/conda-build-test
            shell level : 1
       user config file : /Users/kodegard/.condarc
 populated config files : /Users/kodegard/.condarc
          conda version : 22.11.1
    conda-build version : not installed
         python version : 3.10.8.final.0
       virtual packages : __archspec=1=x86_64
                          __osx=10.16=0
                          __unix=0=0
       base environment : /Users/kodegard/.conda/22.11.1/3.10  (writable)
      conda av data dir : /Users/kodegard/.conda/22.11.1/3.10/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/kodegard/.conda/22.11.1/3.10/pkgs
                          /Users/kodegard/.conda/pkgs
       envs directories : /Users/kodegard/.conda/22.11.1/3.10/envs
                          /Users/kodegard/.conda/envs
               platform : osx-64
             user-agent : conda/22.11.1 requests/2.28.1 CPython/3.10.8 Darwin/21.6.0 OSX/10.16
                UID:GID : 502:20
             netrc file : None
           offline mode : False

Conda Config

==> /Users/kodegard/.condarc <==
auto_update_conda: False
auto_activate_base: False
changeps1: False
channels:
  - defaults

Conda list

# packages in environment at /Users/kodegard/.conda/22.11.1/3.10/envs/conda-build-test:
#
# Name                    Version                   Build  Channel
anaconda-client           1.11.0          py310hecd8cb5_0    defaults
attrs                     22.1.0          py310hecd8cb5_0    defaults
backports                 1.1                pyhd3eb1b0_0    defaults
backports.functools_lru_cache 1.6.4              pyhd3eb1b0_0    defaults
backports.tempfile        1.0                pyhd3eb1b0_1    defaults
backports.weakref         1.0.post1                  py_1    defaults
beautifulsoup4            4.11.1          py310hecd8cb5_0    defaults
blas                      1.0                         mkl    defaults
brotlipy                  0.7.0           py310hca72f7f_1002    defaults
bzip2                     1.0.8                h1de35cc_0    defaults
c-ares                    1.18.1               hca72f7f_0    defaults
ca-certificates           2022.10.11           hecd8cb5_0    defaults
certifi                   2022.12.7       py310hecd8cb5_0    defaults
cffi                      1.15.1          py310h6c40b1e_3    defaults
chardet                   4.0.0           py310hecd8cb5_1003    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
click                     8.0.4           py310hecd8cb5_0    defaults
clyent                    1.2.2           py310hecd8cb5_1    defaults
conda                     22.11.1         py310hecd8cb5_4    defaults
conda-package-handling    2.0.2           py310hecd8cb5_0    defaults
conda-package-streaming   0.7.0           py310hecd8cb5_0    defaults
conda-verify              3.4.2                      py_1    defaults
contextlib2               21.6.0          py310hecd8cb5_0    defaults
coverage                  6.3.2           py310hca72f7f_0    defaults
cryptography              38.0.1          py310hf6deb26_0    defaults
curl                      7.86.0               h6c40b1e_0    defaults
cytoolz                   0.12.0          py310hca72f7f_0    defaults
execnet                   1.9.0              pyhd3eb1b0_0    defaults
expat                     2.4.9                he9d5cce_0    defaults
filelock                  3.9.0           py310hecd8cb5_0    defaults
future                    0.18.2          py310hecd8cb5_1    defaults
gdbm                      1.18                 hdccc71a_4    defaults
gettext                   0.21.0               h7535e17_0    defaults
git                       2.34.1          pl5262h74264fa_0    defaults
glob2                     0.7                pyhd3eb1b0_0    defaults
icu                       58.2                 h0a44026_3    defaults
idna                      3.4             py310hecd8cb5_0    defaults
iniconfig                 1.1.1              pyhd3eb1b0_0    defaults
intel-openmp              2021.4.0          hecd8cb5_3538    defaults
jinja2                    3.1.2           py310hecd8cb5_0    defaults
jsonschema                4.16.0          py310hecd8cb5_0    defaults
jupyter_core              5.1.1           py310hecd8cb5_0    defaults
krb5                      1.19.2               hcd88c3b_0    defaults
libarchive                3.6.1                he336d3b_0    defaults
libcurl                   7.86.0               ha585b31_0    defaults
libcxx                    14.0.6               h9765a3e_0    defaults
libedit                   3.1.20221030         h6c40b1e_0    defaults
libev                     4.33                 h9ed2024_1    defaults
libffi                    3.4.2                hecd8cb5_6    defaults
libiconv                  1.16                 hca72f7f_2    defaults
liblief                   0.12.3               hcec6c5f_0    defaults
libnghttp2                1.46.0               ha29bfda_0    defaults
libssh2                   1.10.0               h0a4fc7d_0    defaults
libxml2                   2.9.14               hbf8cd5e_0    defaults
llvm-openmp               14.0.6               h0dcd299_0    defaults
lz4-c                     1.9.4                hcec6c5f_0    defaults
markupsafe                2.1.1           py310hca72f7f_0    defaults
mkl                       2021.4.0           hecd8cb5_637    defaults
mkl-service               2.4.0           py310hca72f7f_0    defaults
mkl_fft                   1.3.1           py310hf879493_0    defaults
mkl_random                1.2.2           py310hc081a56_0    defaults
nbformat                  5.7.0           py310hecd8cb5_0    defaults
ncurses                   6.3                  hca72f7f_3    defaults
numpy                     1.23.5          py310h9638375_0    defaults
numpy-base                1.23.5          py310ha98c3c9_0    defaults
openssl                   1.1.1s               hca72f7f_0    defaults
packaging                 22.0            py310hecd8cb5_0    defaults
patch                     2.7.6             h1de35cc_1001    defaults
pcre2                     10.37                he7042d7_1    defaults
perl                      5.34.0               h435f0c2_2    defaults
pip                       22.3.1          py310hecd8cb5_0    defaults
pkginfo                   1.8.3           py310hecd8cb5_0    defaults
platformdirs              2.5.2           py310hecd8cb5_0    defaults
pluggy                    1.0.0           py310hecd8cb5_1    defaults
psutil                    5.9.0           py310hca72f7f_0    defaults
py                        1.11.0             pyhd3eb1b0_0    defaults
py-lief                   0.12.3          py310hcec6c5f_0    defaults
pycosat                   0.6.4           py310hca72f7f_0    defaults
pycparser                 2.21               pyhd3eb1b0_0    defaults
pycrypto                  2.6.1           py310hca72f7f_10    defaults
pyflakes                  2.4.0              pyhd3eb1b0_0    defaults
pyopenssl                 22.0.0             pyhd3eb1b0_0    defaults
pyrsistent                0.18.0          py310hca72f7f_0    defaults
pysocks                   1.7.1           py310hecd8cb5_0    defaults
pytest                    7.1.2           py310hecd8cb5_0    defaults
pytest-cov                3.0.0              pyhd3eb1b0_0    defaults
pytest-forked             1.3.0              pyhd3eb1b0_0    defaults
pytest-mock               3.7.0           py310hecd8cb5_0    defaults
pytest-replay             1.4.0              pyhd3eb1b0_0    defaults
pytest-rerunfailures      10.1               pyhd3eb1b0_0    defaults
pytest-xdist              2.5.0              pyhd3eb1b0_0    defaults
python                    3.10.8               h218abb5_1    defaults
python-dateutil           2.8.2              pyhd3eb1b0_0    defaults
python-fastjsonschema     2.16.2          py310hecd8cb5_0    defaults
python-libarchive-c       2.9                pyhd3eb1b0_1    defaults
pytz                      2022.7          py310hecd8cb5_0    defaults
pyyaml                    6.0             py310h6c40b1e_1    defaults
readline                  8.2                  hca72f7f_0    defaults
requests                  2.28.1          py310hecd8cb5_0    defaults
ripgrep                   13.0.0               hc2228c6_0    defaults
ruamel.yaml               0.17.21         py310hca72f7f_0    defaults
ruamel.yaml.clib          0.2.6           py310hca72f7f_1    defaults
setuptools                65.5.0          py310hecd8cb5_0    defaults
shellcheck                0.7.2                hecd8cb5_0    defaults
six                       1.16.0             pyhd3eb1b0_1    defaults
soupsieve                 2.3.2.post1     py310hecd8cb5_0    defaults
sqlite                    3.40.0               h880c91c_0    defaults
tk                        8.6.12               h5d9f67b_0    defaults
toml                      0.10.2             pyhd3eb1b0_0    defaults
tomli                     2.0.1           py310hecd8cb5_0    defaults
toolz                     0.12.0          py310hecd8cb5_0    defaults
tqdm                      4.64.1          py310hecd8cb5_0    defaults
traitlets                 5.7.1           py310hecd8cb5_0    defaults
tzdata                    2022g                h04d1e81_0    defaults
urllib3                   1.26.13         py310hecd8cb5_0    defaults
wheel                     0.37.1             pyhd3eb1b0_0    defaults
xz                        5.2.8                h6c40b1e_0    defaults
yaml                      0.2.5                haf1e3a3_0    defaults
zlib                      1.2.13               h4dc903c_0    defaults
zstandard                 0.18.0          py310hca72f7f_0    defaults
zstd                      1.5.2                hcb37349_0    defaults

Additional Context

No response

dholth commented 1 year ago

Looks like bad test data to me:

% bunzip2 test_index_of_removed_pkg-1.0-1.tar.bz2
bunzip2: test_index_of_removed_pkg-1.0-1.tar.bz2 is not a bzip2 file.

libarchive would extract uncompressed tarfiles named .tar.bz2 but cph 2.0.2 will not.

dholth commented 1 year ago

See conda-build/#4693

We fixed the conda-build test data by compressing it properly.