Platform specific dependencies clarification

[b4rlw]

Not sure if this is a bug, or if I'm misunderstanding the intended functionality. Given the implementation https://github.com/conda/conda-lock/commit/90b370a7b8f2b918820159a3c4f95ac0a31ac446, for the following environment.yml file:

channels:
  - conda-forge
dependencies:
  - python=3.11.*
  - poetry=1.6.*
  - gcc_linux-64
  - gcc_linux-aarch64
platforms:
  - osx-arm64
  - linux-64
  - linux-aarch64

Expected behaviour

If I install the resulting lockfile on:

osx-arm64: I get only Python and Poetry. linux-64: I get Python Poetry and GCC. linux-aarch64: I get Python Poetry and GCC.

Actual behaviour

Locking fails for osx-arm64 because it can't find gcc_linux-64 and gcc_linux-aarch64.

[maresb]

Hey, thanks for the precise report. In order to achieve your expected behavior you should use preprocessing selectors. Note that these are not fully implemented in conda-lock, so feel free to open issues if you run into unexpected behavior.

Here's what your environment.yml should look like:

channels:
  - conda-forge
dependencies:
  - python=3.11.*
  - poetry=1.6.*
  - gcc_linux-64  # [linux]
  - gcc_linux-aarch64  # [aarch64]
platforms:
  - osx-arm64
  - linux-64
  - linux-aarch64

And then when you run conda-lock you should see

WARNING:conda_lock.src_parser.selectors:filtered out line `  - gcc_linux-64  # [linux]` due to unmatchable selector
WARNING:conda_lock.src_parser.selectors:filtered out line `  - gcc_linux-aarch64  # [aarch64]` due to unmatchable selector
WARNING:conda_lock.src_parser.selectors:filtered out line `  - gcc_linux-aarch64  # [aarch64]` due to unmatchable selector
Locking dependencies for ['linux-64', 'linux-aarch64', 'osx-arm64']...
INFO:conda_lock.conda_solver:linux-64 using specs ['python 3.11.*', 'poetry 1.6.*', 'gcc_linux-64']
INFO:conda_lock.conda_solver:linux-aarch64 using specs ['python 3.11.*', 'poetry 1.6.*', 'gcc_linux-64', 'gcc_linux-aarch64']
INFO:conda_lock.conda_solver:osx-arm64 using specs ['python 3.11.*', 'poetry 1.6.*']
 - Install lock using: conda-lock install --name YOURENV conda-lock.yml

(Those warning messages are really confusing, so I opened #498.)

Since platform-specific stuff is a pain, I'd recommend instead using the helper package cxx-compiler which should in most cases get you what you need, regardless of platform.

Does that answer your question?

[b4rlw]

Yes it does, thank you. I didn't understand preprocessing selectors - I saw the words "WARNING" and "unmatchable" and assumed something was going wrong, haha.

In fact cxx-compiler was exactly what I was looking for! My Docker image was failing to build because psutil needed gcc, but wouldn't work using the preprocessing selectors method. Shoving in cxx-compiler and losing the selectors allowed the build to complete properly for all platforms, so cheers for the recommendation!