search

mamba-org / mamba

The Fast Cross-Platform Package Manager
https://mamba.readthedocs.io
BSD 3-Clause "New" or "Revised" License
6.96k stars 359 forks source link

Discussion: micromamba releases should be built and released in this repository #2595

Open pavelzw opened 1 year ago

pavelzw commented 1 year ago

Xref https://github.com/mamba-org/micromamba-releases/issues/5

2536

Right now, the micromamba release workflow is the following:

I think this release workflow is highly inefficient and would propose this alternative:

Like this, we could guarantee that there is always the most recent version available when a release is created.

pavelzw commented 1 year ago

@AntoinePrv @JohanMabille what do you think of this proposal? I could create a PR implementing this in CI if you are on board with it.

jonashaag commented 1 year ago

I strongly support this proposal. The current release process needs too much human interaction and is scattered over multiple repositories. I'm also in favour of hosting the binaries here rather than in a separate repository.

AntoinePrv commented 1 year ago

We have plan for a non statically linked micro-mamba to be the replacement to python mamba so I think we should keep a feedstock and make sure it is up to date.

On push of a tag, the CI builds its own static build versions

Would that be using the recipe from the feedstock?

pavelzw commented 1 year ago

Would that be using the recipe from the feedstock?

I would suggest keeping the recipe for the static build in this repository.

pavelzw commented 1 year ago

I am strongly in favor of a non-statically linked micromamba on conda-forge and deprecating the statically linked version on conda-forge. Usually package manager do dynamic linking, that's what they are here for ;P We can keep the static linking only to this repository.

JohanMabille commented 1 year ago

It would be weird to not have micromamba available on conda-forge. I agree the releasing process is error-prone and implies too many human interactions, but deprecating the micromamba feedstock looks a bit extreme to me.I think we can have both the automatic release on this repo with the process you described, and continue to update the feedstock repo. Ideally we should be able to use a bot / GHA to automagically update the feedstock upon release tag until we get back to a more traditional workflow when we split this repo and the conda-forge bot can detect releases.

pavelzw commented 1 year ago

deprecating the micromamba feedstock looks a bit extreme to me

I agree, maybe it came out wrong. I didn't mean to deprecate the micromamba-feedstock but the static build on conda-forge.

How about switching to a dynamically linked build in the micromamba feedstock (maybe also providing a micromamba-static output)? This would be more in-line with how the other packages are doing it.

The question here would be how many people rely on a static micromamba build being available on anaconda.org/conda-forge/micromamba (for whom such a switch would probably result in an error)... My guess would be not that many, but probably some that depend on a static build being available on

Since mamba-org is in control of all these three domains, this could be migrated without collateral damage.

wolfv commented 1 year ago

I think there are some social aspects we should consider, too:

On the other hand the situation is clearly not perfect - the mix with vcpkg is not great.

One attempt I've done in the past is the boa-forge stuff where we'd maintain all the dependencies needed for micromamba. It has worked pretty well (for all platforms). We could consider building all packages statically from source with something like boa-forge, and that could be potentially done in the feedstock.

pavelzw commented 1 year ago

Btw the new 1.5.2 and 1.5.3 releases are not available on conda-forge yet https://github.com/conda-forge/micromamba-feedstock/issues/162

AntoinePrv commented 1 year ago

We are aware, this is because it requires some changes to the feedstock bu these release don't provide much addition to micromamba/

pavelzw commented 11 months ago

Another issue with building only on conda-forge and not having a clean build+release pipeline only in this repo:

the alpha candidates are now listed in https://api.anaconda.org/release/conda-forge/micromamba/latest

This leads to https://micromamba.snakepit.net/api/micromamba/linux-64/latest and https://micro.mamba.pm/api/micromamba/linux-64/latest providing the alpha releases which should not happen. The https://github.com/mamba-org/micromamba-releases/ repo had the same issue but was fixed in https://github.com/mamba-org/micromamba-releases/commit/fb241b81f520ed00ae7560d10d6f63f96e90369b

If we provided proper release artifacts in this repo, the alpha release would not be necessary.

JohanMabille commented 11 months ago

If we provided proper release artifacts in this repo, the alpha release would not be necessary.

Alpha releases are totally orthgonal to where we host the releases. The problem here is that https://api.anaconda.org/release/conda-forge/micromamba/latest does not filter pre releases, and anaconda does not provide any stable API for getting the latest stable release. And anyway, we need alpha releases so that clients can early test major incompatible changes.

That being said, now that we have the same code base for mamba and (micro)mamba (only the linkage is different), I agree that we should:

I think when this is done, we can archive https://github.com/mamba-org/micromamba-releases

I understand the social aspect of building static mamba on conda-forge, but I think the drawbacks outweigh the advantages.

oursland commented 6 months ago

What's the value-add of a dynamically linked micromamba or mamba?

I am encountering an issue affecting CI builds of a project I am on because the upgrade process of dynamic libraries breaks mamba on Windows: https://github.com/mamba-org/mamba/issues/3292

A statically linked binary would prevent this issue altogether.

jjerphan commented 3 days ago

@oursland: as of 2.0, micromamba is a the statically linked version of mamba with is relevant for CI use-cases.