`foundation` prevents solving with ghcjs

[angerman]

There is a relevant merge request, that has been open for now almost a year.

Can we please amend the cabal file to at least permit building it with ghcjs or trying to build it with ghcjs?

-  if impl(ghc < 8.0) || impl(ghcjs)
+  if impl(ghc < 8.0)
     buildable: False

[angerman]

Just to be clear, the buildable flag outright prohibits this from being solved for. I don't even think I can override buildable through a project.cabal file. So the only option I'd be left with is start a hackage overlay, just to work around this...

[phadej]

Such change cannot be done in revision.

[angerman]

@phadej not? Ohh well. I though revisions could update the full cabal file. :-/

[phadej]

Check https://github.com/haskell-infra/hackage-trustees/blob/master/revisions-information.md#what-can-revisions-change

[angerman]

One more reason to advocate against the use of foundation aggressively then I guess. What a terrible situation.

[endgame]

@angerman You may be able to use source-repository-package or packages (in cabal.project) to select your fork of foundation, so you can avoid this unfortunate constraint without needing your own hackage.

[angerman]

@endgame sure that's what I'm doing, but it's going to be painful to tell everyone to use a forked foundation version if they want to work with ghcjs. The thing is that we are the primary developers of GHCJS right now. It's quite unfortunate that foundation is effectively abandoned and so many people jumped onto the foundation bandwagon back then.

[endgame]

Yes, that's very regrettable. It may be easier to cut foundation out from other packages' build-depends:.

[andreasabel]

@angerman : Your other options are NMU or package takeover: https://github.com/haskell-infra/hackage-trustees/blob/master/policy.md

[Bodigrim]

Can we please amend the cabal file to at least permit building it with ghcjs or trying to build it with ghcjs?

There must be a reproducible evidence first that (starting from a certain version) GHCJS can build foundation, only then trustees may make a revision or NMU. It would not be right to relax constraints speculatively.

It's quite unfortunate that foundation is effectively abandoned and so many people jumped onto the foundation bandwagon back then.

Only a few packages depend on foundation: https://packdeps.haskellers.com/reverse/foundation There was a long delay to update foundation to GHC 9.0, and currently it is broken by GHC 9.2, plus this GHCJS issue... I think it's time for the community to move on, or fork it, or takeover it. Trustees could not be expected to keep it on life support.

[phadej]

@Bodigrim Don''t forget that tls -> cryptonite -> memory depend on foundation (and basement). That said, cryptonite wont work on GHCJS as its all C, but still. These packages are somewhat hard to avoid.

(GHCJS happily builds code with foreign C imports, it just fails at runtime if there are no shims for needed C functions).

[Bodigrim]

@phadej agreed, but it would suffice to fork / take over only two libraries to fix tls, not hundreds of packages.

CC @cdornan on behalf of memory-cd and @kazu-yamamoto on behalf of tls.

[phadej]

TIL about {basement,memory,cryptonite}-cd.

[kazu-yamamoto]

What I know is as follow:

1. Vincent kindly delegated tls to me. So, I'm maintaining it officially.
2. Olivier also stopped using Haskell. Vincent does not delegate cryptonite to anyone. So, nobody maintains cryptonite at this moment.
3. Vincent does not delegate memory, foundation and basement to anybody.

I have received no response from Vincent relating to 1. and 2. My preference is taking over the four packages by volunteers.

[Bodigrim]

@kazu-yamamoto would you be interested to switch tls from cryptonite and memory to cryptonite-cd and memory-cd?

[kazu-yamamoto]

What does -cd stand for? (Anything is possible at this moment.)

[angerman]

Wild guess. -cd stands for @cdornan.

We are making ghcjs C capable with emscripten. This isn't fully automated. You'll still fail at runtime as you'll need to write some bridging logic.

The whole buildable: false logic seems to come from nixpkgs not being able to build it. If that is our gauge, we can throw buildable: false freely around. Same for {c,asm,cmm}-sources.

Let's flip this around: give me an option in the cabal.project file to override nonsense like this in packages and I'll be less annoyed with this. But someone else deciding what I can or cannot do and forcing me to fork stuff is a pretty high barrier.

Again, I know we can work around this kind of issues by building hackage overlays, which would also allow custom patches. I'm just trying to forgo this route as it's a pretty big hammer.

[angerman]

Side note: Vincent and I are colleagues, I wasn't able to reach him recently through $work either 🤷‍♂️ maybe I'm not patient enough?

[endgame]

The whole buildable: false logic seems to come from nixpkgs not being able to build it. If that is our gauge, we can throw buildable: false freely around. Same for {c,asm,cmm}-sources.

I am surprised to hear this, because if nixpkgs fails to build something, they slap a broken = true; on it and move on. (There's a yaml file in nixpkgs which drives (I think) hackage2nix, which generates the derivations.)

[Bodigrim]

@angerman I understand your annoyance, but (for a good reason) Trustees operate in a very conservative mode when it comes to relaxing (as opposite to restricting) bounds.

The whole buildable: false logic seems to come from nixpkgs not being able to build it. If that is our gauge, we can throw buildable: false freely around. Same for {c,asm,cmm}-sources.

Our gauge is that the maintainer prohibited this build configuration. Trustees don't know his reasons and should not speculate about them. If the prohibited configuration is buildable, there should exist a reproducible test evidence, but I see none attached to the PR in question.

Let's flip this around: give me an option in the cabal.project file to override nonsense like this in packages and I'll be less annoyed with this. But someone else deciding what I can or cannot do and forcing me to fork stuff is a pretty high barrier.

Trustees do not maintain or develop Cabal.

[angerman]

@Bodigrim I know. Sorry if this rant was a bit over the line.

[phadej]

Switching tls (alone) to other crypto-provider won't help. E.g. warp depends on x509 suite of libraries (which also depend on cryptonite). There is a large collection of related packages (all by vincent originally?) and forking just few will make sure that any bigger development will still have all of them.

@angerman probably don't need warp to compile with GHCJS, but if warp-tls would depend on both cryptonite-cd (for tls) and cryptonite (for x509)... that would be silly, if it will even work with C-linker and duplicate cbits.

[angerman]

For native platforms it will likely complain about duplicate symbols unless the forks do symbol prefixing (which incidentally some of Vincents) packages actually do.

Yes I believe it's the whole ecosystem of libraries Vincent wrote trying to modernize Haskell from the ground up. That also puts these dependencies naturally at a very dependency important place. His switch from Haskell to rust, without active maintainers for those libraries is problematic.

[angerman]

@phadej's comment gave me an additional idea. How about we allow a substitutes stanza in cabal that allows us to map a dependency to a replacement. Of course the replacement would need to have the same surface api for this to be sensible. But at least the build plans could be computed under this substitution.

Great I'm going out of this thread with two ideas to implement in cabal.

[phadej]

@angerman that's essentially an overlay (package repostory), but more lightweight to setup. I admit I sometimes wished it, but vendoring package version worked so far too. If such package overlay existed, it would encourage fork making, for good and bad.

Bad being people have negative patience nowadays, cryptonite case is well justified, but some other maintainer having a few week holiday isn't. Do we want people forking libraries and uploading them to append-only storage for short-lived needs? I won't like that.

[angerman]

It would basically be backpack at the package level. You wouldn't even need a fork you could have a completely different package just implementing the same surface level api.

One thing I might need to clarify: I'm not the one usually working on haskell packages themselves. My team and I are called into action when stuff doesn't work. So we rarely build packages outright but end up duct-taping stuff left and right. I'm aware my view might be rather different. I care about easily fixing up multiple projects at once.

So far we can get away with patches often as our setup is heavily reliant on nix. This particular package description issue is different as it affect solving prior to actually patching anything and the resolution comes straight from cabal consuming a hackage index. So my only option would be a hackage overlay, which I find very heavy handed.

[phadej]

@angerman the https://cabal.readthedocs.io/en/3.6/installing-packages.html#local-no-index-repositories repositories are very easy to setup: drop tarballs (and optionally a separate .cabal file, which is exactly your need) into a directory, tell cabal to use it, profit.

[angerman]

@phadej ohh that's cool. Slightly less painful than the overlays with the generated index and keys.

Can the url be https as well?

[angerman]

Guess that might bring me back to overlays. Anyway thanks for pointing that out @phadej. I'll need to play with this a little.

[phadej]

Can the url be https as well?

if url https it's the ordinary package repository. There is no https+noindex (as there is no readDirectory for HTTP, so you couldn't even implement it).

[angerman]

Actually, having nix for this, a local directory might work as the result of some derivation that generates it. Oh, this is a cool solution that might just work.

[phadej]

@angerman read the docs well, cabal would like to write cache to that directory, but that can be configured. (Yes, I had nix/readonly locations in mind while implementing that feature).

EDIT: looks like #shared-cache is other way around. It's some time since I made that, so forgot which way is the default. Without it, you probably want to specify some unique (e.g. derivation) name for the repository, so different projects caches don't conflict (if the name was the same, like haskell-nix-overlay)

[angerman]

@phadej thanks. Yes I did read the docs with an eye on read only. And the #shared-cache was what made me think this could actually work.