Formalising our stance regarding binary distribution of cabal-install

[Kleidukos]

I am growing increasingly convinced that the line between us, the upstream development team of Cabal & friends, and the distributors of the cabal-install tool is blurred.

Currently we have the following distribution channels:

• https://downloads.haskell.org/cabal: The "official" bindists produced by our gitlab release pipeline. It's not clear which downstream users benefit from it, as it is a subset of what ghcup distributes;
• GHCup: Through its own channel, distributes a wider variety of bindists, especially for platforms not maintained by the GitLab pipeline;
• Binary Distributions will build their own bindist because they will want to have a say on some lower-level linker & compiler flags, and they also get quite pissed off if they have to ship upstream binaries, see https://wiki.debian.org/Hadoop;
• Source Distributions don't care too much about our bindists.

Looking at these, I have the impression that we're not really using our time correctly in the Cabal team, since our bindists are neither covering all architectures that are requested, nor used by most of distribution channels.

I would like to ask our distributors if this impression is correct, and if there is something the Cabal team can focus on to ease their life, should we disengage from producing bindists ourselves.

---

CC

@juhp for Fedora, @arrowd for FreeBSD, @blackgnezdo for OpenBSD, @depressed-pho for NetBSD, @branchvincent for Brew, @hololeap and @solpeth for Gentoo @rd235 for Debian, @doko42 for Ubuntu, @peti for OpenSUSE, @maralorn for NixOS, @hasufell for GHCup.

(Please indicate if someone else is better suited for your distribution.)

[blackgnezdo]

OpenBSD builds from source using the ports system. We don't even rely on the source tarballs from hackage due to them not having the bootstrap files #7289.

[hololeap]

Gentoo builds Cabal from source as a part of GHC and/or from hackage sources. cabal-install is built purely from hackage sources.

[maralorn]

nixpkgs builds Cabal together with ghc. For that we use the Cabal source distributed as part of ghc. For bootstraping we use ghc bindists which I think include Cabal bindists, which I guess are built by the ghc release pipeline.

We build cabal-install and e.g. other versions of Cabal from hackage sdists.

[angerman]

While not mentioned in the ticket, Haskell.nix also builds from source and never uses binary dists.

[hasufell]

Since cabal upstream has refused to accept my contributions to improve release CI to make it fit GHCup's needs, I stopped relying on it and now build my own binaries.

Further, questionable opinions/practices have reinforced my belief that it's better to not rely on cabal's shipped binaries, given that I have to keep reminding them of botched releases, as well as security backport requests being ignored.

[gbaz]

I think we will find that distribution channels, almost by definition, build their own binaries. Ghcup was an exception and has since changed its practices. That said, I do think binary releases are important as part of what it means to be an upstream maintainer and source-of-truth even if most users get their binaries through other channels.

However, this ticket seems to mainly be focused on acquiring information about who produces their binaries. (all distributions, I think?) and not on what a sensible process is for us as maintainers, and doesn't have what looks to be a particularly worked-through proposal for how we might change.

My view is that the actual production of bindists is not a central source of work to cabal maintainers, but the fact of needing to produce such helps conceptually tie together work that is important. I think that ceasing to produce such binaries could well lead to release management starting to fall apart, as the production of a set of binaries as a "release artifact" is an important unifying concept that helps ensure all other strands of work fit together.

[hasufell]

I agree with @gbaz and would encourage cabal team to just go ahead with whatever binaries they already have produced.

It might make sense to adjust documentation and communicate to end users that these binaries are really just a side effect of the release process and may contain bugs and issues not present when pulling from your distro or building from hackage.

As a result, this would also allow you to minimize the bindists produced to just:

• static alpine linux x86_64
• windows x86_64
• mac aarch64

And drop everything else.

As regards to index state pinning: drop it and make sure you also distribute plan.json (which you already do).

---

Please go ahead with the release and don't delay it further with all these discussions.

[andreabedini]

I wish that producing binary releases wasn't so hard but given the situation I agree we should leave it to more experienced distributors. The time saved can be better used elsewhere.

[peti]

openSUSE does not use cabal-install binaries from upstream. We build those ourselves using the source tarballs available from Hackage.

[arrowd]

OpenBSD builds from source using the ports system. We don't even rely on the source tarballs from hackage due to them not having the bootstrap files https://github.com/haskell/cabal/issues/7289.

Same for FreeBSD

[juhp]

I think this ticket is about cabal-install so no need for people to chime in about ghc Cabal, unless you use non-ghc Cabal which would be interesting info indeed. :-)

[juhp]

Fedora Linux builds cabal-install from Hackage source, of course.

Probably obvious to most other distros, but we/I align cabal-install version with the ghc Cabal version. (Fedora releases also have multiple versions of ghc, but that is not really relevant to this discussion.)

A repeating problem I have noticed in the past is cabal-install not even building with the ghc/Cabal it is naturally released/intended for (of course it builds with cabal-install): mostly just because of silly conservative bounds. This has also impacted Stackage in the past, though the situation seems better lately.

Like many/most Linux distributions we build the official package with Setup not with cabal-install, of course. Though I do have unofficial copr repos of newer cabal-install which are built with cabal-install as a "hack".

(The recent ghc-9.10 alpha1 release is also a pain-point since Hadrian there now needs Cabal-3.10.)

To give some contrast: I believe Fedora Rust uses cargo to build official packages/apps (though Fedora Rust only provides source library packages not binaries, because of too much churn): at times I pondered whether to do the same for Fedora Haskell. Alright probably too much info already. :o)

ps I forget the details, but also this "ghc needs latest cabal-install to work" "nonsense" :grimacing: is annoying too: some of it at least feels like bug which could be fixed in older releases perhaps? Sorry I don't have the details at hand and better to discuss it separately in detail: not sure if a ticket exists. I think if upstream and downstreams spent a bit more time talking and looking to each other these simpler problems could be avoided.

[fendor]

HLS release CI is a consumer of the upstream cabal bindists. Since we produce the binaries for the ghcup vanilla channels, we also use the other tools from the vanilla channel to have a consistent build environment. Dropping the upstream cabal bindists will force us to either compile cabal from source in our release CI, or use cabal from the ghcup main distribution channel, or drop the upstream HLS release binaries.

I think cabal should keep producing release binaries, for the same reason @gbaz argues.

[andreabedini]

Dropping the offical cabal bindists will force us to either compile cabal from source in our release CI, or use cabal from the unofficial ghcup distribution channel, or drop the official HLS release binaries.

That unofficial seems unwarranted in the discussion: ghcup is as official as fedora, nixpkgs or ubuntu. Note that haskell-actions/setup uses ghcup to install cabal so if HLS release CI runs on GHA that's free.

[fendor]

True, and that was not my intention, I will correct this to upstream and ghcup main distribution channel.

[hasufell]

HLS release CI is a consumer of the upstream cabal bindists.

I guess now you see the entire point why I raised the midstream bindist proposal and stopped trying to fix everyone elses release CI: it's impossible.

I suggest for HLS to also just provide the minimum of release binaries with whatever other tools like cabal and GHC support.

GHCup builds its own HLS binaries too and the vanilla channel isn't relevant to most users.

[fishtreesugar]

https://github.com/Homebrew/homebrew-core/blob/master/Formula/c/cabal-install.rb Brew using official bindist to build from source.