Closed tsibley closed 1 year ago
Surfacing this as a PR mostly to socialize the work and solicit feedback on any and all parts of it.
I've got the following problem:
mamba install -n nextstrain-base -c "nextstrain/label/pull-1" nextstrain-base
Looking for: ['nextstrain-base']
bioconda/osx-64 No change
conda-forge/noarch No change
pkgs/r/osx-64 No change
pkgs/r/noarch No change
pkgs/main/osx-64 No change
pkgs/main/noarch No change
nextstrain/label/pull-1/osx-64 No change
nextstrain/label/pull-1/noarch No change
conda-forge/osx-64 23.8MB @ 4.8MB/s 5.5s
bioconda/noarch 3.7MB @ 599.9kB/s 6.0s
Encountered problems while solving:
- nothing provides _libgcc_mutex ==0.1 conda_forge needed by nextstrain-base-20221017T235849Z-hb0f4dca_1_locked
Hmm. I wonder if that is a platform-specific issue, i.e. if _libgcc_mutex ==0.1
from conda-forge isn't available in the osx-64
subdir? Although CI on macOS passes…
If it is platform-specific, then we might need to build platform-specific versions of nextstrain-base
, even though it contains no code itself. Or alternatively, figure out how to robustly add platform selectors to the platform-specific deps.
The _libgcc_mutex
package is available for Linux:
$ mamba search --override-channels -c conda-forge --subdir linux-64 '_libgcc_mutex'
Loading channels: done
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_libgcc_mutex 0.1 free conda-forge
_libgcc_mutex 0.1 main conda-forge
but apparently not macOS:
$ mamba search --override-channels -c conda-forge --subdir osx-64 '_libgcc_mutex'
Loading channels: done
No match found for: _libgcc_mutex. Search: *_libgcc_mutex*
The following packages are not available from current channels:
- _libgcc_mutex
Current channels:
- https://conda.anaconda.org/conda-forge/osx-64
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
PackagesNotFoundError: The following packages are not available from current channels:
- _libgcc_mutex
Current channels:
- https://conda.anaconda.org/conda-forge/osx-64
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Though now I'm unclear on why install works on macOS in CI?
Ah, CI is misleading because it's installing the unlocked first-pass package on macOS. Bah! I'll fix that up, but it means there's definitely some more work to do to make this package work on macOS.
This isn't an issue on Linux in CI (or locally), which I'd checked. I think the difference may come down to file mtime (possibly affected by artifact upload/download) or directory entry collation on macOS filesystems. In any case, CI should make sure the locked version is what's tested/installed.
CI's now failing for macOS, so that's progress. ;-)
Ok, I reworked this to build separate packages for Conda's linux-64
and osx-64
platforms instead of a noarch
package. CI is testing the correct packages on each platform and passes. I compared the generated locked/recipe.yaml
files for each platform: there's a slew of packages used by only one platform, and nearly all the build ids for the packages are different (this makes sense, in retrospect).
Installing again from the nextstrain/label/pull-1
channel should now work on macOS. Give it a shot?
Since this is a brand new repo as of yet unused by anything, I'm going to merge this initial implementation. I would still appreciate review from folks and will address any changes as subsequent PRs.
Merging this will allow easier testing of https://github.com/nextstrain/cli/pull/228, which is itself part of a broader feature—the Conda runtime—that's also as of yet unused by anything.
See the README and individual commit messages.
Related issue(s)
Todo
locked/recipe.yaml
in version control? If so, then CI should commit it back. If not, then remove the current copy from git. Answer: ditch it from version control, but make sure it's part of CI artifacts for troubleshooting. It's also already embedded in the Conda packages themselves, for introspection if necessary._libgcc_mutex
Testing