Closed ysangkok closed 1 year ago
@ysangkok, thanks for reporting. The following are some notes for me, to help me investigate:
For GHC 9.4.2, the default setup-info
dictionary used by Stack currently has for Linux/x86_64:
linux64
ghc-9.4.2-x86_64-deb9-linux.tar.xzlinux64-tinfo6
ghc-9.4.2-x86_64-fedora33-linux.tar.xzIn addition the GHC developers also publish for GHC 9.4.2:
tinfo6
is provided with:
libgmp10
is provided with:
On libc6
versions:
ubuntu-latest
on GitHub Actions) comes with 2.31, Ubuntu 22.04 ('jammy') comes with 2.35On Stack's current behviour:
tinfo6
GHC variant because it has detected libtinfo.so.6
and libgmp.so.10
(given the logic of Stack.Setup.getGhcBuilds
). Stack looks to https://downloads.haskell.org/~ghc/9.4.2/ghc-9.4.2-x86_64-fedora33-linux.tar.xz for the linux64-tinfo6
GHC variant.So, if I understand correctly, it would be more inclusive for linux64-tinfo6
to look to ghc-9.4.2-x86_64-deb10-linux.tar.xz rather than ghc-9.4.2-x86_64-fedora33-linux.tar.xz.
@hasufell Ok, I installed 9.4.2 through ghcup and used system-ghc: true
to make Stack use this GHC. This serves as a usable workaround for the time being, thanks! Unsure how I'd achieve it with the new experimental hook that it prompts for during installation. Because even though I answered yes to that question, system-ghc: true
seems to be necessary.
@hasufell Ok, I installed 9.4.2 through ghcup and used
system-ghc: true
to make Stack use this GHC. This serves as a usable workaround for the time being, thanks! Unsure how I'd achieve it with the new experimental hook that it prompts for during installation. Because even though I answered yes to that question,system-ghc: true
seems to be necessary.
If you set system-ghc to true, then the hook is ignored. Refer to the documentation: https://docs.haskellstack.org/en/stable/yaml_configuration/#ghc-installation-customisation-experimental
It's also possible you hit this bug: https://github.com/commercialhaskell/stack/issues/5888#issuecomment-1263123244
@hasufell, thanks for the link to setup-info on commercialhaskell/stackage-content/pull/117 (I am trying to centralize discussion here though)
It seems overkill to replace the whole setup-info dictionary. But luckily Stack allows for configuring the ghc-build
(not variant
!). So I just added ghc-build: standard
to ~/.stack/config.yaml
and that works on Bullseye for GHC 9.4.3.
"Standard" is somewhat confusingly listed as the default ghc-build in the documentation. The same section also mentions the autodetection, which specifying the build explicitly seems to disable (which is what I want).
It seems that the ghc-build
options that the documentation list don't correspond to what GHC actually publishes: values like fedora33
seen in https://downloads.haskell.org/ghc/9.4.3/ . Wouldn't it be better if the ghc-build
options just corresponded to what was is actually available from the GHC team?
The underlying internal value is already just a string.
So I'd argue that in stack-setup-2.yaml
instead of
linux64-nopie:
url: "https://downloads.haskell.org/~ghc/9.4.3/ghc-9.4.3-x86_64-fedora33-linux.tar.xz"
we should have
linux64-fedora33:
url: "https://downloads.haskell.org/~ghc/9.4.3/ghc-9.4.3-x86_64-fedora33-linux.tar.xz"
One could imagine the YAML file describing which dynamic libraries the different builds depend on, but that seems really complex and too much work.
The Stack sources themselves actually suggests using a static build, but probably there would be some opposition to that (judging from previous discussions on static vs dynamic linking in the general FOSS sphere). I wouldn't be opposed to that though.
So two proposals:
ghc-build
options should equal the fragments used in GHC build filenames. E.g. fedora33
instead of nopie
What do you think of these proposals @hasufell and @mpilgrem (and lurkers)? I'd implement it if you want me to. Unsure whether it means stack-setup-3.yaml
or not.
proposal 1: ghc-build options should equal the fragments used in GHC build filenames. E.g. fedora33 instead of nopie
That's basically what GHCup does, see: https://gitlab.haskell.org/haskell/ghcup-hs/-/commit/e924ad8278966795ea6f8393fc38900c549f6e2a
This would allow you to fake a distro and then get the bindists that are known to work on said distro.
Then you use stack installation hooks to defer GHC installation to ghcup: https://docs.haskellstack.org/en/stable/yaml_configuration/#ghc-installation-customisation
Alternatively, just overwrite the bindist URLs for your own distro: https://gitlab.haskell.org/haskell/ghcup-hs/-/blob/master/data/config.yaml#L45
Stacks concept is different and depends on shared libraries. Both have pros and cons, but as explained, you can already mix them.
proposal 2: remove dynamic library version logic in Stack, just use the Alpine build
Afair GHC devs are not sure this is a good idea yet. We need more confidence that static builds don't pose problems.
FYI I ran into this problem in GitHub actions using GHC 9.4.3 (stack snapshot nightly-2022-11-19
, not using the setup-haskell
GitHub action) with ubuntu-latest
. It seems to be fixed by using ubuntu-22.04
instead.
@brandonchinn178, agreed - Pantry ran into this in the summer and applied the same fix. See - https://github.com/commercialhaskell/pantry/blob/master/.github/workflows/tests.yml.
This is the current ghcup metadata: https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-0.0.7.yaml
Ubuntu 20.04 ('focal' - ubuntu-latest on GitHub Actions)
By 15 December 2022, ubuntu-latest
will refer to 22.04 for larger runners. Source: github.blog
There is a discussion of the problem from the GHC perspective here: https://gitlab.haskell.org/ghc/ghc/-/issues/22268. It has been floated that the GHC developers might revert to providing also a Fedora 27 version from GHC 9.4.5.
@ysangkok, has explained in more detail why using the Debian 10 version of GHC 9.4.x is irksome for users of other Linux distributions in this Slack thread: https://haskell-foundation.slack.com/archives/C022RSL1M8F/p1670542821095849. So, https://github.com/commercialhaskell/stackage-content/pull/117 does seem to be a dead end.
I assume the 'pure Stack' solution for the current releases of Stack (that is, one that does not involve letting Stack use GHCup to manage GHC, using the 'hook') is to use Stack's ghc-variant
(with a custom 'variant') and setup-info
keys in its global YAML configuration file - as described in https://haskell-foundation.slack.com/archives/C022RSL1M8F/p1670542821095849 and https://docs.haskellstack.org/en/stable/yaml_configuration/#setup-info. While this was originally intended to accommodate 'cutting edge' Linux distributions, it is not limited to that.
The slack links are not public. One needs to sign in to view the contents. I don't have an account. Please copy paste the relevant parts.
@ysangkok, has explained in more detail why using the Debian 10 version of GHC 9.4.x is irksome for users of other Linux distributions
I learnt about these problems from a link to hasufell's post. I merely reproduced the warnings, which look like this:
/ghc-9.4.3-deb10/lib/ghc-9.4.3/bin/./ghc-9.4.3: /lib64/libtinfo.so.6: no version information available (required by /ghc-9.4.3-deb10/lib/ghc-9.4.3/bin/../lib/x86_64-linux-ghc-9.4.3/libHShaskeline-0.8.2-ghc9.4.3.so)
I assume the 'pure Stack' solution for the current releases of Stack (that is, one that does not involve letting Stack use GHCup to manage GHC, using the 'hook') is to use Stack's ghc-variant
As far as I know, the ghc-variant is not the issue (I did initially think it was, though, when I posted this issue, which is why it is mentioned above). I worked around this issue by adjusting the ghc-build
(not the variant), which is also what my proposed patch #5982 affects.
It is correct that a custom setup-info would allow even the tinfo6 ghc-build to point to the Debian variant, and in that case no Stack changes would be needed, since the heuristics would choose a working binary even without using the standard
build. But as I see it, there is already a name for the Debian ghc-build, and it is standard
aka linux64
. Stack should use the "Debian" build when on Debian. I am aiming to only fix the issue at hand, no more.
The slack links are not public. One needs to sign in to view the contents. I don't have an account. Please copy paste the relevant parts.
I have pasted it in a Gist.
Some links where not copied over, they were:
In summary, I propose using the os-release library that hasufell is maintaining, and GHCup is using. The fallback methods that GHCup also has don't seem necessary, especially not for just detecting Debian derivatives, which is what I am aiming for.
I think we need to discuss this: https://github.com/haskell/ghcup-hs/issues/719
The knowledge about these things needs to be centralized.
I'd be ok with adding support for parsing and installing stack flavored metadata and let the user decide in the end which method works best.
I'm not sure if mixing them together is a good idea, to be honest (as in: using distro+version AND library keys like tinfo5
).
The other project I've been thinking about is stop relying on upstream GHC bindists and simply building them for an arbitrary number of distros. I already do that for HLS: https://github.com/haskell/haskell-language-server/pull/3406/files#diff-e426ed45842837026e10e66af23d9c7077e89eacbe6958ce7cb991130ad05ada
So the idea is ghcup and stack projects maintain their own unofficial bindist set. This really needs to be coordinated though, because otherwise we'll run into ABI mismatch errors with prebuilt HLS binaries all day long.
Since we just got self hosted darwin M1 runners for github and have plans to extend our capacity for the github haskell namespace, this could be feasible.
CCing @david-christiansen
So, it is far from ideal, but if you are on Debian/Ubuntu, you want to add the following to your Stack global YAML configuration file (config.yaml
) (eg for GHC 9.4.3):
setup-info:
ghc:
linux64-tinfo6:
9.4.3:
url: "https://downloads.haskell.org/~ghc/9.4.3/ghc-9.4.3-x86_64-deb10-linux.tar.xz"
or, if you also want the safety of checking that what is at that URL is what is expected:
setup-info:
ghc:
linux64-tinfo6:
9.4.3:
url: "https://downloads.haskell.org/~ghc/9.4.3/ghc-9.4.3-x86_64-deb10-linux.tar.xz"
content-length: 184837364
sha1: 28a94f2da6077d725a7d72f48bc909cf9f2444e3
sha256: 940ac2b1770dc63b5f3f38f829bfe69f4a572d6b26cd93094cdd99d5300b5067
(EDIT: That will cause Stack to prefer the 'Debian 10' version of GHC 9.4.3 to the 'default' tinfo6
version of GHC 9.4.3. EDIT2: Once Stack has fetched that version of GHC 9.4.3, it only needs to fetch it once; the setup-info
is then redundant and can be deleted.)
I have tested that on Ubuntu 22.04 (on WSL).
Picking up a comment above:
But as I see it, there is already a name for the Debian ghc-build, and it is
standard
akalinux64
.
As I understand it, standard
does not refer to 'Debian' as such. It refers to any Linux distribution that has libtinfo.so.5
and libgmp.so.5
. The GHC developers provide a version of GHC that works on such distributions, namely the 'Debian 9' version of GHC. standard
seems to me to be a misnomer in 2022.
It seems to me that the easiest immediate solution is to rework Stack to make use of an extra 'tag' that distinguishes Linux 'tinfo6' distributions that can support libc6
2.32 (eg Fedora 33) from other Linux 'tinfo6' distributions that can support versions of libc6
up to 2.26 but not 2.32 (eg Fedora 27, Debian 10 and 11). eg linux64-tinfo6-libc6-2.32
v plain linux64-tinfo6
.
I've raised pull request #5987 to implement a version of my idea above, but I am dependent on users of Linux distributions that cannot support libc6
2.32 to test/comment on it.
I've switched things around, so that linux64-tinfo6
is used for Linux distributions that can support libc6 2.32
(which includes Ubuntu 22.04, which is now ubuntu-latest
on GitHub-hosted runners; and /Debian 12 (bookworm)) and linux64-tinfo6-libc6-pre2.32
is used for Linux distributions that can only support libc6
that are less than 2.32. My reasoning is that (a) linux64-tinfo6-libc6-pre2.32
will fall away as versions of Linux distributions catch up with libc6
2.32 and (b) a lot of CI will be using ubuntu-latest
and have relied on the existing linux64-tinfo6
.
I have merged my idea into the master
branch version of Stack (eg stack upgrade --source-only --git
), to allow it to be tested more easily in practice. If it turns out not to be viable, my changes can be reversed.
I changed the key from linux64-tinfo6-libc6-pre2.32
to linux64-tinfo6-libc6-pre232
because, it turns out, Pantry requires the key to be a valid Cabal package name. Cabal package names have to be one or more alphanumeric words, separated by hyphens, each word containing at least one letter.
In case it helps anyone using Stack through Nix:
I was having similar issues when I was doing nix-shell -p stack
(or more precisely nix-shell nix-shell -I nixpkgs=/path/to/gitrepos/NixOS/nixpkgs -p stack
(to get the exact nixpkgs commit I want)) to get into a shell with stack
and using it from there.
The problem disappeared when, instead of creating a shell, I started doing nix profile install nixpkgs#stack
(or more precisely nix profile install /path/to/gitrepos/NixOS/nixpkgs#stack
(to get the exact nixpkgs commit I want)) to get stack
available globally and using it from there.
WORKAROUND
ghc-build: standard
to~/.stack/config.yaml
apt install libtinfo5
Steps to reproduce
docker run -it debian:bullseye
apt update && apt install curl
curl -sSL https://get.haskellstack.org/ | sh
stack new --resolver nightly
stack.yaml
to saycompiler: ghc-9.4.2
stack run
Expected
I expected to see some dependency failure because the current nightly snapshot probably doesn't work on GHC-9.4.2.
Actual
Fails during installation of the Fedora 33 build of GHC:
Seems like there are GHC releases for various versions of Debian. Debian 9 is referenced in stack-setup.yaml for the
linux64
variant, maybe it works with that one. (EDIT: It does, this is what the workaround at the top does) But I think it should work out of the box. I don't think I can use theghc-variant
option to select a Debian build, since it seems I can only append tolinux64-tinfo6
. (EDIT: Use ghc-build, not variant) The build I'd like to try is named e.g.linux64
in thestack-setup.yaml
. Seems like it could be useful to have a flag similar toghc-variant
that would allow me to get rid oftinfo6
.Stack version
Method of installation