LANG: dependency testing for python 3.9 updates to metagenome distro

[lizgehret]

hey @misialq 👋

i'm trying to figure out how to get the metagenome distribution to solve with python 3.9, so just playing around with some dependency changes. i know pysam is a req for assembly's generate-reads action, but i'm just trying to nail down where the environment solve failures are coming from. opening a PR on this named branch will allow for the PR i have open in distributions to test out these changes 🙂

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.33%. Comparing base (02fb91a) to head (1d87e60).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #80 +/- ## ======================================= Coverage 98.33% 98.33% ======================================= Files 27 27 Lines 1802 1802 ======================================= Hits 1772 1772 Misses 30 30 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[misialq]

Hey @lizgehret, sure thing, sounds good! Let me know whenever I can help! :)

[lizgehret]

Hey @misialq! So here's a status update for the metagenome distro:

pysam is quite problematic for solving the environment - particularly because it depends on libdeflate which has a max pin of 1.19 for the current build. Because of this, including pysam in the environment upgrades/downgrades a bunch of other dependencies in our q2-plugins (see this doc for full list), which render the metapackage unsolvable.

I've opened up a PR in bioconda to ask for them to relax their pin on libdeflate for pysam - which, if they accept, will be the easiest solution (and quite reasonable because they are then working with a more recent version of libdeflate, which almost all of our packages depend on).

Long story short, I think there are a few options here:

1. Wait to hear back from bioconda to see if they will solve this on their end.
2. Leave pysam out of the metapackage for this release, and advise any users of the generate_reads action that they'll need to pip install pysam separately (i.e. we can't include it in the metapackage but they'll need to install separately to utilize that particular action).
3. Rip off the code from ISS that requires pysam in the first place. This will require the most work out of all three options, so I'm not sure if this is a great idea.

We honestly might be fine to go with option 2 if the bioconda folks aren't amenable to this update. Since metagenome distro is technically still in alpha, pip installing one dependency for one action isn't a completely unreasonable thing to do.

[misialq]

Hey @lizgehret, thanks so much for investigating this 🙏 Your plan sounds good to me - should we then just wait a little bit to see whether they are ok with that and if not we proceed with plan 2? If yes, is there a "time box" you'd like to set so that we don't wait forever? 😆

[lizgehret]

Good question @misialq - maybe we plan to move forward with option 2 if we don't hear from them by the end of next week? Otherwise our May release could turn into another n+1 situation 😅

[misialq]

Sounds good! EONW it is - let's hope for the best 😆

[lizgehret]

Hey @misialq! No word from bioconda unfortunately, so let's move forward with option two. Want to get this PR merged and then I'll kick off an official Prepare PR after that?

Note/reminder (mainly for myself): I'll make sure to include info about this in our release notes so that folks know to install pysam via pip if they're trying to run the generate_reads action using the 2024.5 metagenome distro environment.

[misialq]

Hey @lizgehret! Sounds good! Let me know if there's anything else I can do to help move this along 💪