Release 23.10 - Githubissues

jakirkham commented 8 months ago

Now that Conda 23.10 is released and the blogpost is up. Congratulations to the team for making that happen 👏

What would be our next steps for packaging it in Miniforge?

hmaarrfk commented 8 months ago

we used to have troubles with boa compatibility.

if the tests pass. I'm ok with a new release

jakirkham commented 8 months ago

Yes this is a good point

Raised this item at a previous core meeting, the feedback we got there was to switch to using conda-build + conda-libmamba-solver, which ships (and is default) in this release

So maybe this provides a way forward?

cc @jaimergp @wolfv (in case you have anything to add here)

hmaarrfk commented 8 months ago

i'm ok with removing the boa test. A year ago, without it, it would cause a bunch of downgrades to happen during the build process, which can sometimes be problematic.

isuruf commented 8 months ago

The plan me and @jaimergp talked about is to first release conda-smithy which will switch to conda-build+conda-libmamba-solver by default, wait to see if there are any issues and if there are no issues, release a new mambaforge.

isuruf commented 8 months ago

The other solution was to wait for a new boa release. See https://github.com/mamba-org/boa/issues/381

hmaarrfk commented 8 months ago

great sounds awesome!

jakirkham commented 8 months ago

Thanks Isuru! 🙏

This helps answer the question of what the steps would be

first release conda-smithy which will switch to conda-build+conda-libmamba-solver by default, wait to see if there are any issues and if there are no issues, release a new mambaforge

If we want to do more testing first, that sounds reasonable

The other solution was to wait for a new boa release. See https://github.com/mamba-org/boa/issues/381

If we think there will be movement there, great!

That said, it looks like that issue has been open for a week without response

Also this was brought up at the core meeting a few weeks ago ( after months of being in a holding pattern around various incompatibilities: https://github.com/conda-forge/boa-feedstock/pull/73 )

To be clear this is not intended as criticism towards anyone. There have been lots of changes in tooling and it takes work to keep things in a compatible state

That said, think taking a practical approach to resolving this (like moving forward here and letting boa catch up when they are able) may be a better approach for all involved. We can always add back the boa test once things are in a working state

jaimergp commented 8 months ago

The plan me and @jaimergp talked about is to first release conda-smithy

Yep! Anything holding back a conda-smithy release this week?

jakirkham commented 8 months ago

Updating the thread, conda-smithy release 3.28.0 is packaged ( https://github.com/conda-forge/conda-smithy-feedstock/pull/267 )

hmaarrfk commented 8 months ago

I just really don't understand the constraints of boa. it says it can support newer mamba, but then the Miniforge builder complains https://github.com/conda-forge/boa-feedstock/blob/main/recipe/meta.yaml#L27

isuruf commented 8 months ago

@hmaarrfk, the latest builds were marked broken due to an issue with building recipes on conda-forge.

jakirkham commented 8 months ago

Yep the PR that made that change has a comment at the end that provides more context: https://github.com/conda-forge/boa-feedstock/pull/74#issuecomment-1711469618

Agree the situation around boa is confusing. Have been confused myself at times in trying to understand what is going on there

That said, as the newest conda defaults to libmamba as the solver and conda-build will use it too. Just switching to conda-build should give us a path forward in the interim

This approach was tested in staged-recipes and is now being tested in feedstocks with the latest conda-smithy. If all goes smoothly, we can use a similar approach here

jakirkham commented 7 months ago

cc @jezdez (for awareness)

jaimergp commented 7 months ago

The problem right now is that the latest conda-build is not compatible with boa. And if we update the Miniforge distro here with the latest stuff, boa will cause a conflict when it's installed in-feedstock.

That's not a problem if we switch to conda-build + conda-libmamba-solver. IN fact, we have done so I think? But some feedstocks are incompatible due to a bug I'm chasing in https://github.com/conda/conda-libmamba-solver/issues/386. These feedstocks need boa for mambabuild.

So I see three solutions:

Make boa compatible with the latest conda-build. This just needs a release, but Wolf detect some last minute bugs so not sure what's going to take.
Make the Miniforge version configurable in conda-forge.yml. Instead of defaulting to latest we can let users pick an old one. This would allow us to release a new Miniforge with 23.10.
We wait until I fix the CLS bug.

boa users would need to pin to 23.3 or something via conda-forge.yml. e.g.

conda_build_tool: mambabuild
conda_install_tool: mamba
miniforge_version: 23.3.1-0  # this key doesn't exist yet

Thoughts?

hmaarrfk commented 7 months ago

Make boa compatible with the latest conda-build. This just needs a release, but Wolf detect some last minute bugs so not sure what's going to take.

I feel like we should choose this. though it is unclear to me exactly what bugs exist.

they can't be irreconcilable....

jakirkham commented 7 months ago

Thanks for summarizing the situation Jaime! 🙏

That is also my understanding. Hope that helps for other folks running into this issue without context

While in principle I understand why folks would want boa to be compatible, it is practically out-of-our-hands

In a prior core meeting, we discussed options and decided to drop the boa dependency (Wolf supported this)

The next step we decided was to start using conda-build with the libmamba solver more broadly (as Isuru explained above https://github.com/conda-forge/miniforge/issues/511#issuecomment-1797452725 ). That way we can test and find issues

We have been doing that for a little bit. We found a few bugs (expected given we went from testing in staged-recipes to testing in all of conda-forge). The conda-libmamba-solver team has been prompt in resolving them

A new Miniforge would use the new solver by default. So think we want to be confident that is working as expected. See the current bug as just part of that effort we had committed to

Think once we are comfortable with the conda-libmamba-solver situation. We can go ahead with a release

To summarize:

If 1 happens, great! Just don't think we should block on that any more

If we need to go faster for some reason, would also be ok with 2, but would need to be convinced as to why we need to go faster (since this will also result in an uptick in Miniforge user issues)

Would much prefer 3. It is consistent with what we have done so far and should minimize disruption (while letting us to continue to make progress)

h-vetinari commented 7 months ago

Make boa compatible with the latest conda-build. It needs a release, but Wolf detect some last minute bugs so not sure what's going to take.

Presumably that release was 0.16? In any case, both before and after https://github.com/conda-forge/boa-feedstock/pull/77, the boa recipe did not specify a cap on the conda-build version.

hmaarrfk commented 7 months ago

Reopenning since #518 was merged in haste and will likely be reverted (or at the very least, not marked as the latest release)

conda-forge / miniforge

Release 23.10 #511