conda / conda-build

Commands and tools for building conda packages
https://docs.conda.io/projects/conda-build/
Other
379 stars 420 forks source link

Building packages with epochs twice fails #5372

Open isuruf opened 3 months ago

isuruf commented 3 months ago

Checklist

What happened?

Building the following twice fails

package:
  name: test
  version: "1!1.0.0"

test:
  commands:
    - echo 1

Error:

``` (base) jcmorin-mac-anac:aggregate jcmorin$ conda-build jcmorin/test-feedstock WARNING: No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.26 Adding in variants from internal_defaults Adding in variants from /Users/jcmorin/anaenv/aggregate/conda_build_config.yaml Adding in variants from /Users/jcmorin/anaenv/aggregate/jcmorin/test-feedstock/recipe/conda_build_config.yaml Attempting to finalize metadata for test BUILD START: ['test-1!1.0.0-hca03da5_0.tar.bz2'] source tree in: /Users/jcmorin/miniconda/conda-bld/test_1717627300292/work Packaging test Packaging test-1!1.0.0-hca03da5_0 WARNING: No files or script found for output test number of files: 0 Fixing permissions INFO :: Time taken to mark (prefix) 0 replacements in 0 files was 0.00 seconds TEST START: /Users/jcmorin/miniconda/conda-bld/osx-arm64/test-1!1.0.0-hca03da5_0.tar.bz2 Renaming work directory '/Users/jcmorin/miniconda/conda-bld/test_1717627300292/work' to '/Users/jcmorin/miniconda/conda-bld/test_1717627300292/work_moved_test-1!1.0.0-hca03da5_0_osx-arm64' shutil.move(work)=/Users/jcmorin/miniconda/conda-bld/test_1717627300292/work, dest=/Users/jcmorin/miniconda/conda-bld/test_1717627300292/work_moved_test-1!1.0.0-hca03da5_0_osx-arm64) Reloading output folder (local): ...working... done Solving environment (_test_env): ...working... done ## Package Plan ## environment location: /Users/jcmorin/miniconda/conda-bld/test_1717627300292/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold The following NEW packages will be INSTALLED: test: 1!1.0.0-hca03da5_0 local Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done export PREFIX=/Users/jcmorin/miniconda/conda-bld/test_1717627300292/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold export SRC_DIR=/Users/jcmorin/miniconda/conda-bld/test_1717627300292/test_tmp + echo 1 + exit 0 1 Resource usage statistics from testing test: Process count: 2 CPU time: Sys=0:00:00.0, User=0:00:00.0 Memory: 13.3M Disk usage: 24B Time elapsed: 0:00:02.0 TEST END: /Users/jcmorin/miniconda/conda-bld/osx-arm64/test-1!1.0.0-hca03da5_0.tar.bz2 Renaming work directory '/Users/jcmorin/miniconda/conda-bld/test_1717627300292/work' to '/Users/jcmorin/miniconda/conda-bld/test_1717627300292/work_moved_test-1!1.0.0-hca03da5_0_osx-arm64_main_build_loop' shutil.move(work)=/Users/jcmorin/miniconda/conda-bld/test_1717627300292/work, dest=/Users/jcmorin/miniconda/conda-bld/test_1717627300292/work_moved_test-1!1.0.0-hca03da5_0_osx-arm64_main_build_loop) # Automatic uploading is disabled # If you want to upload package(s) to anaconda.org later, type: # To have conda build upload to anaconda.org automatically, use # conda config --set anaconda_upload yes anaconda upload \ /Users/jcmorin/miniconda/conda-bld/osx-arm64/test-1!1.0.0-hca03da5_0.tar.bz2 anaconda_upload is not set. Not uploading wheels: [] INFO :: The inputs making up the hashes for the built packages are as follows: { "test-1!1.0.0-hca03da5_0.tar.bz2": { "recipe": { "channel_targets": "defaults", "target_platform": "osx-arm64" } } } #################################################################################### Resource usage summary: Total time: 0:00:06.9 CPU usage: sys=0:00:00.0, user=0:00:00.0 Maximum memory usage observed: 13.3M Total disk usage observed (not including envs): 24B #################################################################################### Source and build intermediates have been left in /Users/jcmorin/miniconda/conda-bld. There are currently 2 accumulated. To remove them, you can run the ```conda build purge``` command (base) jcmorin-mac-anac:aggregate jcmorin$ (base) jcmorin-mac-anac:aggregate jcmorin$ conda-build jcmorin/test-feedstock WARNING: No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.26 Adding in variants from internal_defaults Adding in variants from /Users/jcmorin/anaenv/aggregate/conda_build_config.yaml Adding in variants from /Users/jcmorin/anaenv/aggregate/jcmorin/test-feedstock/recipe/conda_build_config.yaml Attempting to finalize metadata for test BUILD START: ['test-1!1.0.0-hca03da5_0.tar.bz2'] source tree in: /Users/jcmorin/miniconda/conda-bld/test_1717627309286/work Packaging test Packaging test-1!1.0.0-hca03da5_0 WARNING: No files or script found for output test number of files: 0 Fixing permissions INFO :: Time taken to mark (prefix) 0 replacements in 0 files was 0.00 seconds TEST START: /Users/jcmorin/miniconda/conda-bld/osx-arm64/test-1!1.0.0-hca03da5_0.tar.bz2 Traceback (most recent call last): File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/version.py", line 44, in __call__ return cls._cache_[arg] ~~~~~~~~~~~^^^^^ KeyError: '!1.0.0-hca03da5_0' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/match_spec.py", line 60, in __call__ return super().__call__(**parsed) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/match_spec.py", line 190, in __init__ self._match_components = self._build_components(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/match_spec.py", line 425, in _build_components return frozendict(_make_component(key, value) for key, value in kwargs.items()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/frozendict/_frozendict_py.py", line 55, in __new__ dict.__init__(self, *args, **kwargs) File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/match_spec.py", line 425, in return frozendict(_make_component(key, value) for key, value in kwargs.items()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/match_spec.py", line 439, in _make_component matcher = _implementors[field_name](value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/version.py", line 46, in __call__ val = cls._cache_[arg] = super().__call__(arg) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/version.py", line 523, in __init__ vspec_str, matcher, is_exact = self.get_matcher(vspec) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/version.py", line 552, in get_matcher raise InvalidVersionSpec(vspec_str, "invalid operator") conda.exceptions.InvalidVersionSpec: Invalid version '!1.0.0-hca03da5_0': invalid operator The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/Users/jcmorin/miniconda/bin/conda-build", line 11, in sys.exit(execute()) ^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda_build/cli/main_build.py", line 590, in execute api.build( File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda_build/api.py", line 250, in build return build_tree( ^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda_build/build.py", line 3653, in build_tree test(pkg, config=metadata.config.copy(), stats=stats) File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda_build/build.py", line 3270, in test environ.clean_pkg_cache(metadata.dist(), metadata.config) File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda_build/environ.py", line 1211, in clean_pkg_cache for cache_pkg_id in package_cache.query(dist): ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/core/package_cache_data.py", line 156, in query param = MatchSpec(param) ^^^^^^^^^^^^^^^^ File "/Users/jcmorin/miniconda/lib/python3.12/site-packages/conda/models/match_spec.py", line 82, in __call__ raise InvalidMatchSpec(msg, details=e) from e conda.exceptions.InvalidMatchSpec: Invalid spec 'test-1!1.0.0-hca03da5_0': Invalid version '!1.0.0-hca03da5_0': invalid operator ```

Thanks to @JeanChristopheMorinPerso for the reproducer and the following analysis

It works if I downgrade to conda-index 0.3. Downgrading that also downgrades conda to 24.1.2 and conda-build to 24.1.1, so not sure which downgrade fixes it

Found it. I tried to re-upgrade everything and slowly downgrade conda to 24.1 and I don't see the problem. I ten slowly downgraded conda-build one version after the other and the problem started to appear in 24.1.2

At least on defaults

cc @jaimergp

Conda Info

From @JeanChristopheMorinPerso

          conda version : 24.5.0
    conda-build version : 24.5.1

Conda Config

No response

Conda list

No response

Additional Context

No response

isuruf commented 1 month ago

Any updates on this?