Support multi-file version constraints

[knedlsepp]

Description

This adds support for multi-file version constraints.

Prior work: conda/conda-lock#300

[netlify[bot]]

✅ Deploy Preview for conda-lock ready!

Name	Link
🔨 Latest commit	83792713bcd09e91a09811d2e05e0f6f60ef3e1c
🔍 Latest deploy log	https://app.netlify.com/sites/conda-lock/deploys/656211a5b4a7740008f848c2
😎 Deploy Preview	https://deploy-preview-546--conda-lock.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[maresb]

Please don't be shy about cleaning up the existing code if you're so inclined.

[knedlsepp]

Please don't be shy about cleaning up the existing code if you're so inclined.

:+1: Although the draft is not me being shy, but rather clumsily trying to make it work at all. ;-)

[knedlsepp]

Status update: Versions seem to work. I'm having issues however when a build is specified.

Real world example:

This would work

name: environment
dependencies:
  - openblas=*
  - openblas=0.3.20

(Also when split across files)

The following doesn't work:

name: environment
dependencies:
  - openblas=*=openmp*
  - openblas=0.3.20=openmp_h53a8fd6_1

[maresb]

Very nice. Any idea why the build number is problematic?

[knedlsepp]

Very nice. Any idea why the build number is problematic?

Apparently in a conda MatchSpec it is possible to have versions consisting of multiple "constraints", but it is currently not possible to specify build consisting of multiple "constraints". (I found this: https://github.com/conda/conda/pull/11612 )

$ conda search "_openmp_mutex=*=*_llvm,2_kmp_llvm"
InvalidVersionSpec: Invalid version '*=*_llvm': invalid character(s)

$ conda search "_openmp_mutex=*=2_kmp_llvm,*_llvm"
InvalidVersionSpec: Invalid version '*=2_kmp_llvm': invalid character(s)

$ conda search _openmp_mutex[build=*_llvm,2_kmp_llvm] # Part after "," is ignored
Loading channels: - 
done
# Name                       Version           Build  Channel             
_openmp_mutex                    4.5          0_llvm  conda-forge         
_openmp_mutex                    4.5          1_llvm  conda-forge         
_openmp_mutex                    4.5      2_kmp_llvm  conda-forge         

$ conda search _openmp_mutex[build=2_kmp_llvm,*_llvm] # Part after "," is ignored
Loading channels: done
# Name                       Version           Build  Channel             
_openmp_mutex                    4.5      2_kmp_llvm  conda-forge         

$ conda search _openmp_mutex[build=*_llvm,build=2_kmp_llvm] # Later bracket argument is used
Loading channels: done
# Name                       Version           Build  Channel             
_openmp_mutex                    4.5      2_kmp_llvm  conda-forge         

$ conda search _openmp_mutex[build=2_kmp_llvm,build=*_llvm]  # Later bracket argument is used
Loading channels: - 
done
# Name                       Version           Build  Channel             
_openmp_mutex                    4.5          0_llvm  conda-forge         
_openmp_mutex                    4.5          1_llvm  conda-forge         
_openmp_mutex                    4.5      2_kmp_llvm  conda-forge    

$ conda search _openmp_mutex=*=2_kmp_llvm[build=*_llvm] # bracket argument overwrites earlier argument
Loading channels: done
# Name                       Version           Build  Channel             
_openmp_mutex                    4.5          0_llvm  conda-forge         
_openmp_mutex                    4.5          1_llvm  conda-forge         
_openmp_mutex                    4.5      2_kmp_llvm  conda-forge         

The commit I pushed now deals with that by (ab-?)using fnmatch.fnmatchcase to merge the two build strings ("2_kmp_llvm", "*_llvm") into "2_kmp_llvm".

It's working nicely for my real world usage to work around the issue in https://github.com/conda/conda-lock/issues/386. For that I render a conda-env-pinned-tmp.yml from an existing lockfile, then I add my new package to conda-env.yml then I run conda-lock using both these files.

conda-lock render --kind env --filename-template conda-env-pinned-tmp
sed -i 's/ --hash/ # --hash/g' conda-env-pinned-tmp.yml # packaging.requirements.InvalidRequirement: Expected end or semicolon (after version specifier)
conda-lock lock --conda `which conda` --file ./conda-env-pinned-tmp.yml --file ./conda-env.yml --lockfile conda-lock.yml --platform linux-64

So now I guess we need some more tests, but I'm optimistic that the current solution is a nice improvement. :)

[knedlsepp]

@maresb I think this is now good enough for a first review. The tests aimed at the build part of the matchspec are mostly covering my personal use-cases of specifying mpi and openmp build variants. I did not yet test if this would work for the kind of "overlapping globs" mentioned in https://github.com/conda/conda/pull/11612.

[maresb]

Thanks @knedlsepp! I will try to review this soon. Once I get to it, I hope to release it as v2.6.0.

[knedlsepp]

Thanks for the review. I did apply your suggestions. Still need to look closer into the category -> categories change.