search

nextstrain / conda-base

Conda package build for nextstrain-base
https://anaconda.org/Nextstrain/nextstrain-base
1 stars 1 forks source link

Initial implementation #1

Closed tsibley closed 1 year ago

tsibley commented 1 year ago

See the README and individual commit messages.

Related issue(s)

Todo

Testing

tsibley commented 1 year ago

Surfacing this as a PR mostly to socialize the work and solicit feedback on any and all parts of it.

corneliusroemer commented 1 year ago

I've got the following problem:

mamba install -n nextstrain-base -c "nextstrain/label/pull-1" nextstrain-base 

Looking for: ['nextstrain-base']

bioconda/osx-64                                               No change
conda-forge/noarch                                            No change
pkgs/r/osx-64                                                 No change
pkgs/r/noarch                                                 No change
pkgs/main/osx-64                                              No change
pkgs/main/noarch                                              No change
nextstrain/label/pull-1/osx-64                                No change
nextstrain/label/pull-1/noarch                                No change
conda-forge/osx-64                                  23.8MB @   4.8MB/s  5.5s
bioconda/noarch                                      3.7MB @ 599.9kB/s  6.0s
Encountered problems while solving:
  - nothing provides _libgcc_mutex ==0.1 conda_forge needed by nextstrain-base-20221017T235849Z-hb0f4dca_1_locked
tsibley commented 1 year ago

Hmm. I wonder if that is a platform-specific issue, i.e. if _libgcc_mutex ==0.1 from conda-forge isn't available in the osx-64 subdir? Although CI on macOS passes…

tsibley commented 1 year ago

If it is platform-specific, then we might need to build platform-specific versions of nextstrain-base, even though it contains no code itself. Or alternatively, figure out how to robustly add platform selectors to the platform-specific deps.

tsibley commented 1 year ago

The _libgcc_mutex package is available for Linux:

$ mamba search --override-channels -c conda-forge --subdir linux-64 '_libgcc_mutex'
Loading channels: done
# Name                       Version           Build  Channel             
_libgcc_mutex                    0.1     conda_forge  conda-forge         
_libgcc_mutex                    0.1            free  conda-forge         
_libgcc_mutex                    0.1            main  conda-forge

but apparently not macOS:

$ mamba search --override-channels -c conda-forge --subdir osx-64 '_libgcc_mutex'
Loading channels: done
No match found for: _libgcc_mutex. Search: *_libgcc_mutex*
The following packages are not available from current channels:

  - _libgcc_mutex

Current channels:

  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

PackagesNotFoundError: The following packages are not available from current channels:

  - _libgcc_mutex

Current channels:

  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Though now I'm unclear on why install works on macOS in CI?

tsibley commented 1 year ago

Ah, CI is misleading because it's installing the unlocked first-pass package on macOS. Bah! I'll fix that up, but it means there's definitely some more work to do to make this package work on macOS.

This isn't an issue on Linux in CI (or locally), which I'd checked. I think the difference may come down to file mtime (possibly affected by artifact upload/download) or directory entry collation on macOS filesystems. In any case, CI should make sure the locked version is what's tested/installed.

tsibley commented 1 year ago

CI's now failing for macOS, so that's progress. ;-)

tsibley commented 1 year ago

Ok, I reworked this to build separate packages for Conda's linux-64 and osx-64 platforms instead of a noarch package. CI is testing the correct packages on each platform and passes. I compared the generated locked/recipe.yaml files for each platform: there's a slew of packages used by only one platform, and nearly all the build ids for the packages are different (this makes sense, in retrospect).

Installing again from the nextstrain/label/pull-1 channel should now work on macOS. Give it a shot?

tsibley commented 1 year ago

Since this is a brand new repo as of yet unused by anything, I'm going to merge this initial implementation. I would still appreciate review from folks and will address any changes as subsequent PRs.

Merging this will allow easier testing of https://github.com/nextstrain/cli/pull/228, which is itself part of a broader feature—the Conda runtime—that's also as of yet unused by anything.