search

nextstrain / conda-base

Conda package build for nextstrain-base
https://anaconda.org/Nextstrain/nextstrain-base
1 stars 1 forks source link

ENH: Add lockfile with exact versions of the various packages for inspection somewhere in this repo #6

Closed corneliusroemer closed 1 year ago

corneliusroemer commented 1 year ago

I installed this meta-package using

mamba create -n nextstrain-base -c nextstrain nextstrain-base

and was surprised to find iqtree 1.6 instead of a current version. I'm curious whether this is just a thing that happens to me or whether everyone gets this old package.

Is there such a thing as a lockfile for this metapackage or does this not exist? How can we check which versions get installed then, and make sure there are no hickups? Maybe we need to add constraints to prevent this from happening? Like specifying iqtree >=2 etc?

Here's the env I got: env.txt

tsibley commented 1 year ago

Is there such a thing as a lockfile for this metapackage or does this not exist?

The meta-package is the "lockfile". Each generated and published package locks its full dependency tree. The src/recipe.yaml file in version control is the basis for the initial shallow dependencies.

(Have you read thru the README?)

How can we check which versions get installed then, and make sure there are no hickups?

Inspect the generated Conda packages. Note that dependencies vary between platforms (linux-64, osx-64).

You can use micromamba search:

micromamba search --json --override-channels -c nextstrain nextstrain-base \
  | jq '.result.pkgs[0] | {url, depends}'

or Anaconda's API:

curl -fsSL https://api.anaconda.org/release/nextstrain/nextstrain-base/latest \
  | jq '.distributions | map(select(.attrs.subdir == "linux-64")) | .[0] | {basename, depends: .attrs.depends}'

or look at the info/recipe/recipe.yaml.template included in each package as metadata:

# Download a .conda package archive one way or another, then…
micromamba package extract ~/Downloads/nextstrain-base-20221109T175138Z-hb0f4dca_1_locked.conda /tmp/pkg
cat /tmp/pkg/info/recipe/recipe.yaml.template

or look at this repo's CI build logs or artifacts.

Maybe we need to add constraints to prevent this from happening? Like specifying iqtree >=2 etc?

Maybe! We should see why the resolver uses iqtree 1 in the first place.

tsibley commented 1 year ago

Expanding on what the README implies: the fully-locked recipes aren't tracked in version control because:

  1. They are platform-specific, so we'd have to track one per platform.
  2. It's not clear how to handle branches/merges/tags to keep updates from conflicting.
  3. Doing so would be noisy in the commit logs.
  4. The most authoritative thing (the thing that matters) is what's in the actual generated (and published) packages, so inspecting those seems best.
  5. It makes the build workflow is very akin to https://github.com/nextstrain/docker-base. The src/recipe.yaml file here is to the Dockerfile there as the generated Conda packages here are to the generated Docker images there.
corneliusroemer commented 1 year ago

Thanks! I hadn't fully digested the README now it's all clear. Downloaded and inspected the lockfile from a CI run and see that IQtree 1.6 is in there.

It could be very helpful to have your first comment above on "various way to inspect the lockfile" somewhere above the develop section in the README, as part of "how to test/verify".