InvalidSpec error for spec ">=0.4,<0.4.4" in sktime

tm-slavik57 commented 5 months ago

Checklist

[X] I added a descriptive title
[X] I searched open reports and couldn't find a duplicate

What happened?

When creating an environment it fails with:

Collecting package metadata (repodata.json): / WARNING conda.core.index:push_record(234): Skipping conda-forge/win-64::sktime-0.28.0-py310h5588dad_0 due to InvalidSpec: >=0.4,<0.4.4,
WARNING conda.core.index:push_record(234): Skipping conda-forge/win-64::sktime-0.28.0-py311h1ea47a8_0 due to InvalidSpec: >=0.4,<0.4.4,
WARNING conda.core.index:push_record(234): Skipping conda-forge/win-64::sktime-0.28.0-py38haa244fe_0 due to InvalidSpec: >=0.4,<0.4.4,
WARNING conda.core.index:push_record(234): Skipping conda-forge/win-64::sktime-0.28.0-py39hcbf5309_0 due to InvalidSpec: >=0.4,<0.4.4,
done
Solving environment: failed

InvalidVersionSpec: Invalid version '>=0.4,<0.4.4,': cannot join single expression

The minimal environment file to reproduce:

name: CondaInvalidSpecDebug
channels:
  - conda-forge

dependencies:
  - sktime=0.28.0

I ran the command:

conda env create --file environment.yml

Conda Info

active environment : None
            shell level : 0
       user config file : C:\Users\Slavas\.condarc
 populated config files : C:\Users\Slavas\.condarc
          conda version : 24.3.0
    conda-build version : not installed
         python version : 3.8.18.final.0
                 solver : classic
       virtual packages : __archspec=1=skylake
                          __conda=24.3.0=0
                          __win=0=0
       base environment : C:\Users\Slavas\Miniconda3  (writable)
      conda av data dir : C:\Users\Slavas\Miniconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\Slavas\Miniconda3\pkgs
                          C:\Users\Slavas\.conda\pkgs
                          C:\Users\Slavas\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\Slavas\Miniconda3\envs
                          C:\Users\Slavas\.conda\envs
                          C:\Users\Slavas\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/24.3.0 requests/2.31.0 CPython/3.8.18 Windows/10 Windows/10.0.19045
          administrator : False
             netrc file : None
           offline mode : False

Conda Config

==> C:\Users\Slavas\.condarc <==
allow_softlinks: True
allow_non_channel_urls: True
channel_priority: flexible
channels:
  - conda-forge
  - defaults
solver: classic

Conda list

# packages in environment at C:\Users\Slavas\Miniconda3:
#
# Name                    Version                   Build  Channel
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
boltons                   24.0.0             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0            py38hd3f51b4_1    conda-forge
bzip2                     1.0.8                hcfcfb64_5    conda-forge
ca-certificates           2024.2.2             h56e8100_0    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cffi                      1.16.0           py38h91455d4_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     24.3.0           py38haa244fe_0    conda-forge
conda-libmamba-solver     24.1.0             pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
console_shortcut          0.1.1                         4    defaults
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
fmt                       10.2.1               h181d51b_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               2.4              py38haa244fe_3    conda-forge
krb5                      1.21.2               heb0366b_0    conda-forge
libarchive                3.7.2                h313118b_1    conda-forge
libcurl                   8.7.1                hd5e4a3a_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
libmamba                  1.5.8                h3f09ed1_0    conda-forge
libmambapy                1.5.8            py38h9d63bcc_0    conda-forge
libsolv                   0.7.28               h12be248_2    conda-forge
libsqlite                 3.45.2               hcfcfb64_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libxml2                   2.12.6               hc3477c8_1    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              he774522_1000    conda-forge
menuinst                  2.0.2            py38hd3f51b4_0    conda-forge
openssl                   3.2.1                hcfcfb64_1    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.4.0              pyhd8ed1ab_0    conda-forge
powershell_shortcut       0.0.1                         3    defaults
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6            py38h91455d4_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.8.18               h1aa4202_0    defaults
python_abi                3.8                      2_cp38    conda-forge
reproc                    14.2.4.post0         hcfcfb64_1    conda-forge
reproc-cpp                14.2.4.post0         h63175ca_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
ruamel.yaml               0.18.6           py38h91455d4_0    conda-forge
ruamel.yaml.clib          0.2.8            py38h91455d4_0    conda-forge
setuptools                69.5.1             pyhd8ed1ab_0    conda-forge
sqlite                    3.45.2               hcfcfb64_0    conda-forge
tqdm                      4.66.2             pyhd8ed1ab_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
vc                        14.3                hcf57466_18    conda-forge
vc14_runtime              14.38.33130         h82b7239_18    conda-forge
vs2015_runtime            14.38.33130         hcb4865c_18    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml-cpp                  0.8.0                h63175ca_0    conda-forge
zstandard                 0.22.0           py38hea8d35e_0    conda-forge
zstd                      1.5.5                h12be248_0    conda-forge

Additional Context

The only thing I found in the sktime repo related to the error is this line which seems legit:

  "pycatch22>=0.4,<0.4.4",

https://github.com/sktime/sktime/blob/31dd08f10a942f9034a71c0bcf3bffc4e158cb57/pyproject.toml#L213C3-L213C27

Perhaps the 22 in the pycatch22 package name messes things up 🤷

Edit It happens only on windows, on linux it works fine

tm-slavik57 commented 5 months ago

Seems like it's a bug with the sktime conda package recipe, it uploaded the dependencies for windows with an extra comma , for the pycatch22 library.

('pykalman >=0.9.5', 'pmdarima >=1.8.0,!=1.8.1,<3.0.0', 'stumpy >=1.5.1', 'pycatch22 >=0.4,<0.4.4,', 'prophet >=1.1', 'scikit-posthocs >=0.6.5', 'filterpy >=1.4.5', 'pystan ==2.19.1.1', 'pyod >=0.8.0', 'statsmodels >=0.12.1', 'tbats >=1.1.0', 'hmmlearn >=0.2.7', 'numba >=0.53', 'tslearn >=0.5.2,<0.7.0', 'seaborn >=0.11.0', 'kotsu >=0.3.1', 'tsfresh >=0.17.0', 'gluonts >=0.9.0', 'esig ==0.9.7', 'statsforecast >=0.5.2,<1.8.0', 'matplotlib-base >=3.3.2')

It would be great to add a protection against such cases.

If there is an issue with the dependencies of a package then I would print a log about the library name and the issue with the dependency. But, if the specific package version is not required then I wouldn't fail the entire resolution, I would skip the problematic version

fkiraly commented 5 months ago

To add a few things here:

the recipe is on conda-forge, so it may have to do with conda-forge rather than with conda. I am not an expert on either, just wanted to raise this.
the offending conda-forge recipe is in this PR: https://github.com/conda-forge/sktime-feedstock/pull/117
it did pass all the automated tests, for multiple operating systems including windows

travishathaway commented 5 months ago

@tm-slavik57,

Just curious, is this issue happening with the libmamba solver, the classic solver or both?

To test, you could run the install commands with either --solver=libmamba or --solver=classic to see which one is affected.

tm-slavik57 commented 5 months ago

@tm-slavik57,

Just curious, is this issue happening with the libmamba solver, the classic solver or both?

To test, you could run the install commands with either --solver=libmamba or --solver=classic to see which one is affected.

@travishathaway Running with libmamba solver worked, running with the classic solver failed as described above

fkiraly commented 5 months ago

I see!

So: 1. it is a conda issue after all, and 2. this explains it why, at 50-100.000 conda downloads a month, no one has complained earlier...

travishathaway commented 5 months ago

@tm-slavik57,

Thanks for reporting back. Currently, the preferred work around for this issue will be to upgrade to the libmamba solver. I bring this issue up with the other maintainers to see if this is something we want to consider working on.

tm-slavik57 commented 5 months ago

@travishathaway For now I patched the package on conda-forge: https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/pull/706 So for now there is no issue to work around, but I think it is still worth solving

kenodegard commented 4 months ago

Moved to conda-build so we can look into adding MatchSpec linters to check for bad specs when building packages.

conda / conda-build