Alex generates code that cannot be compiled by GHC 9.2

Vekhir commented 1 year ago

Hi, the issue occurs with the package hopenpgp-tools (Hackage, source). The code works fine with GHC 9.0.2, but fails to build with GHC 9.2.8 with the errors below. They concern some changes with IntN# requirements in the type signatures. It looks similar to #187, but occurs on both alex 3.2.7.4 and 3.4.0.0 as released on Hackage.

I also made a report over at hopenpgp-tools: clint/hopenpgp-tools#6

Edit: Solution

Older versions of alex generate code incompatible with GHC 9.2, which is fixed in newer versions. However, packaging issues can prevent alex from regenerating code from incompatible versions. Make sure that the package does not include build artifacts.

System information

OS: Arch Linux Kernel: Linux 6.5.7-arch1-1 GHC: 9.2.8 hopenpgp: 2.9.8 hopenpgp-tools: 0.23.7 alex: 3.2.7.4, 3.4.0.0

Error log:

Preprocessing executable 'hkt' for hopenpgp-tools-0.23.7..
Building executable 'hkt' for hopenpgp-tools-0.23.7..
[1 of 5] Compiling HOpenPGP.Tools.Lexer ( dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs, dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.dyn_o )

dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs:1463:50: error:
    • Couldn't match expected type ‘Int16#’ with actual type ‘Int#’
    • In the fifth argument of ‘alex_scan_tkn’, namely ‘sc’
      In the expression:
        alex_scan_tkn user__ input__ 0# input__ sc AlexNone
      In the expression:
        case alex_scan_tkn user__ input__ 0# input__ sc AlexNone of
          (AlexNone, input__')
            -> case alexGetByte input__ of
                 Nothing -> AlexEOF
                 Just _ -> AlexError input__'
          (AlexLastSkip input__'' len, _) -> AlexSkip input__'' len
          (AlexLastAcc k input__''' len, _)
            -> AlexToken input__''' len (alex_actions ! k)
     |
1463 |   = case alex_scan_tkn user__ input__ 0# input__ sc AlexNone of
     |                                                  ^^

dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs:1493:1: error:
    Couldn't match type ‘Int#’ with ‘Int16#’
    Expected: t
              -> t1
              -> Int#
              -> AlexInput
              -> Int16#
              -> AlexLastAcc
              -> (AlexLastAcc, AlexInput)
      Actual: t
              -> t1
              -> Int#
              -> AlexInput
              -> Int#
              -> AlexLastAcc
              -> (AlexLastAcc, AlexInput)
     |
1493 | alex_scan_tkn user__ orig_input len input__ s last_acc =
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...

dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs:1508:27: error:
    • Couldn't match expected type ‘Int#’ with actual type ‘Int32#’
    • In the first argument of ‘(+#)’, namely ‘base’
      In the expression: base +# ord_c
      In an equation for ‘offset’: offset = (base +# ord_c)
     |
1508 |                 offset = (base +# ord_c)
     |                           ^^^^

dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs:1511:73: error:
    • Couldn't match expected type ‘Int#’ with actual type ‘Int16#’
    • In the first argument of ‘(==#)’, namely ‘check’
      In the first argument of ‘tagToEnum#’, namely ‘(check ==# ord_c)’
      In the second argument of ‘(&&)’, namely
        ‘(tagToEnum# (check ==# ord_c))’
     |
1511 |                 new_s = if GTE(offset,0#) && EQ(check,ord_c)
| ^^^^^

dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs:1516:13: error:
    • Couldn't match expected type ‘Int16#’ with actual type ‘Int#’
    • In the pattern: -1#
      In a case alternative: -1# -> (new_acc, input__)
      In the expression:
        case new_s of
          -1# -> (new_acc, input__)
          _ -> alex_scan_tkn
                 user__ orig_input
                 (if c < 0x80 || c >= 0xC0 then (len +# 1#) else len) new_input
                 new_s new_acc
     |
1516 |             -1# -> (new_acc, input__)
     |             ^^^

andreasabel commented 1 year ago

Quick question @Vekhir, out of curiousity : Is there any reason why you are using 0.23.7 and not the latest release, 0.23.8 of hopenpgp-tools.

Vekhir commented 1 year ago

@andreasabel There are three main reasons:

The official Arch package is on 0.23.7 and updating is another variable if something goes wrong
hopenpgp-tools 0.23.8 only introduces support for optparse-applicative 0.18, so the update doesn't fix the issue
0.23.8 requires optparse-applicative >=0.18.1.0 and updating to that would introduce more variables while not fixing the issue (the official Arch package is on 0.17.0.0).

In short, there is nothing to suggest that 0.23.8 fixes this issue, so I stick with the version that the official Arch package uses.

andreasabel commented 1 year ago

Thanks for the answer!

I checked building 0.23.7 with GHC 9.2.8 and everything went fine:

$ cabal get hopenpgp-tools-0.23.7
...
$ cd hopenpgp-tools-0.23.7/
$ cabal build -w ghc-9.2.8
...
Preprocessing executable 'hkt' for hopenpgp-tools-0.23.7...
unused terminals: 2
shift/reduce conflicts:  12
Building executable 'hkt' for hopenpgp-tools-0.23.7...
[1 of 5] Compiling HOpenPGP.Tools.Lexer ( /Users/abel/tmp/hopenpgp-tools-0.23.7/dist-newstyle/build/x86_64-osx/ghc-9.2.8/hopenpgp-tools-0.23.7/x/hkt/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs, /Users/abel/tmp/hopenpgp-tools-0.23.7/dist-newstyle/build/x86_64-osx/ghc-9.2.8/hopenpgp-tools-0.23.7/x/hkt/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.o )
...
Linking ...

Could it be that you have some HOpenPGP/Tools/Lexer.hs generated by an older version of alex? Probably not. It is more likely you have a stale alex in your PATH. What is which alex?

But now I see that you might be using the v1 interface of cabal.

[1 of 5] Compiling HOpenPGP.Tools.Lexer ( dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs

Try using cabal v2-build (which is cabal build with any recent Cabal).

Vekhir commented 1 year ago

The package is built in a clean chroot, i.e. in a newly created, isolated environment and according to the instructions in the PKGBUILD.

This means that not cabal build, but runhaskell Setup build is used. Does this make a difference?

Output of which alex: /usr/bin/alex

andreasabel commented 1 year ago

Output of which alex: /usr/bin/alex

It could well be that this alex is picked up, you can check alex --version whether it is too old.

I suppose runhaskell Setup build also accepts the -v3 verbosity flag, so maybe this will print the PATH it uses etc and which alex is used.

I have no experience with runhaskell Setup instead of cabal, seems like a thing from the past. Or maybe, if you do not want to install cabal-install, but just Cabal, then you might do so.

andreasabel commented 1 year ago

I don't think this is a problem of alex but more one of your setup. So, I am closing this. We can continue the discussion anyway.

Vekhir commented 1 year ago

I've collected some more information which might be useful.

Output of alex --version: Alex version 3.4.0.0, (c) 2003 Chris Dornan and Simon Marlow
Running with -v3 shows that it uses /usr/bin/alex, see below
dist/build/hot/hot-tmp/HOpenPGP/Tools/Lexer.hs
dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs
hopenpgp-tools.cabal for 0.23.7, as available on Hackage.

Interesting result

Interestingly enough, the former "hot" version does contain the #if __GLASGOW_HASKELL__ >= 901 guards, while the "hkt" version does not. Considering that this was one of the tell-tale signs of #187, it seems like the configuration is correct for hot, but somehow different for hkt.

What do you think?

Info

Running with -v3, i.e. runhaskell Setup build -v3 -j17:

Configuring hopenpgp-tools-0.23.7...
Component build order: executable 'hot', executable 'hokey', executable 'hkt',
executable 'hop'
Environment: [("BUILDTOOL","devtools"),("BUILDTOOLVER","1:1.0.4-1-any"),("CFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection"),("CHOST","x86_64-pc-linux-gnu"),("COMMAND_MODE","legacy"),("CXXFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS"),("DEBUGINFOD_URLS","https://debuginfod.archlinux.org "),("HOME","/build"),("LANG","C.UTF-8"),("LDFLAGS","-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"),("LOGNAME","builduser"),("MAIL","/var/mail/builduser"),("MAKEFLAGS","-j17"),("OLDPWD","/build/hopenpgp-tools/src"),("PATH","/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl"),("PWD","/build/hopenpgp-tools/src/hopenpgp-tools-0.23.7"),("PYTHONHASHSEED","0"),("SHELL","/bin/bash"),("SHLVL","1"),("SOURCE_DATE_EPOCH","1697665707"),("SUDO_COMMAND","/bin/bash -c bash -c cd\\ /startdir;\\ makepkg\\ \"$@\" -bash --syncdeps --noconfirm --log --holdver --skipinteg"),("SUDO_GID","0"),("SUDO_UID","0"),("SUDO_USER","root"),("TERM","xterm-256color"),("TEXTDOMAIN","pacman-scripts"),("TEXTDOMAINDIR","/usr/share/locale"),("USER","builduser")]
/usr/bin/ghc-pkg-9.2.8 init dist/package.conf.inplace -v2
GHC package manager version 9.2.8
writing cache dist/package.conf.inplace/package.cache
creating dist/build/hot
creating dist/build/hot/autogen
creating dist/build/hot/autogen
Preprocessing executable 'hot' for hopenpgp-tools-0.23.7..
creating dist/build/hot/hot-tmp/HOpenPGP/Tools
Environment: [("BUILDTOOL","devtools"),("BUILDTOOLVER","1:1.0.4-1-any"),("CFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection"),("CHOST","x86_64-pc-linux-gnu"),("COMMAND_MODE","legacy"),("CXXFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS"),("DEBUGINFOD_URLS","https://debuginfod.archlinux.org "),("HOME","/build"),("LANG","C.UTF-8"),("LDFLAGS","-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"),("LOGNAME","builduser"),("MAIL","/var/mail/builduser"),("MAKEFLAGS","-j17"),("OLDPWD","/build/hopenpgp-tools/src"),("PATH","/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl"),("PWD","/build/hopenpgp-tools/src/hopenpgp-tools-0.23.7"),("PYTHONHASHSEED","0"),("SHELL","/bin/bash"),("SHLVL","1"),("SOURCE_DATE_EPOCH","1697665707"),("SUDO_COMMAND","/bin/bash -c bash -c cd\\ /startdir;\\ makepkg\\ \"$@\" -bash --syncdeps --noconfirm --log --holdver --skipinteg"),("SUDO_GID","0"),("SUDO_UID","0"),("SUDO_USER","root"),("TERM","xterm-256color"),("TEXTDOMAIN","pacman-scripts"),("TEXTDOMAINDIR","/usr/share/locale"),("USER","builduser")]
/usr/bin/alex -g -o dist/build/hot/hot-tmp/HOpenPGP/Tools/Lexer.hs HOpenPGP/Tools/Lexer.x
creating dist/build/hot/hot-tmp/HOpenPGP/Tools
Environment: [("BUILDTOOL","devtools"),("BUILDTOOLVER","1:1.0.4-1-any"),("CFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection"),("CHOST","x86_64-pc-linux-gnu"),("COMMAND_MODE","legacy"),("CXXFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS"),("DEBUGINFOD_URLS","https://debuginfod.archlinux.org "),("HOME","/build"),("LANG","C.UTF-8"),("LDFLAGS","-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"),("LOGNAME","builduser"),("MAIL","/var/mail/builduser"),("MAKEFLAGS","-j17"),("OLDPWD","/build/hopenpgp-tools/src"),("PATH","/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl"),("PWD","/build/hopenpgp-tools/src/hopenpgp-tools-0.23.7"),("PYTHONHASHSEED","0"),("SHELL","/bin/bash"),("SHLVL","1"),("SOURCE_DATE_EPOCH","1697665707"),("SUDO_COMMAND","/bin/bash -c bash -c cd\\ /startdir;\\ makepkg\\ \"$@\" -bash --syncdeps --noconfirm --log --holdver --skipinteg"),("SUDO_GID","0"),("SUDO_UID","0"),("SUDO_USER","root"),("TERM","xterm-256color"),("TEXTDOMAIN","pacman-scripts"),("TEXTDOMAINDIR","/usr/share/locale"),("USER","builduser")]
/usr/bin/happy -agc -o dist/build/hot/hot-tmp/HOpenPGP/Tools/Parser.hs HOpenPGP/Tools/Parser.y
unused terminals: 2
shift/reduce conflicts:  12
Building executable 'hot' for hopenpgp-tools-0.23.7..
creating dist/build/hot
creating dist/build/hot/hot-tmp
Environment: [("BUILDTOOL","devtools"),("BUILDTOOLVER","1:1.0.4-1-any"),("CFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection"),("CHOST","x86_64-pc-linux-gnu"),("COMMAND_MODE","legacy"),("CXXFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS"),("DEBUGINFOD_URLS","https://debuginfod.archlinux.org "),("HOME","/build"),("LANG","C.UTF-8"),("LDFLAGS","-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"),("LOGNAME","builduser"),("MAIL","/var/mail/builduser"),("MAKEFLAGS","-j17"),("OLDPWD","/build/hopenpgp-tools/src"),("PATH","/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl"),("PWD","/build/hopenpgp-tools/src/hopenpgp-tools-0.23.7"),("PYTHONHASHSEED","0"),("SHELL","/bin/bash"),("SHLVL","1"),("SOURCE_DATE_EPOCH","1697665707"),("SUDO_COMMAND","/bin/bash -c bash -c cd\\ /startdir;\\ makepkg\\ \"$@\" -bash --syncdeps --noconfirm --log --holdver --skipinteg"),("SUDO_GID","0"),("SUDO_UID","0"),("SUDO_USER","root"),("TERM","xterm-256color"),("TEXTDOMAIN","pacman-scripts"),("TEXTDOMAINDIR","/usr/share/locale"),("USER","builduser")]
/usr/bin/ghc --make -no-link -fbuilding-cabal-package -O -j17 -dynamic -fPIC -osuf dyn_o -hisuf dyn_hi -outputdir dist/build/hot/hot-tmp -odir dist/build/hot/hot-tmp -hidir dist/build/hot/hot-tmp -stubdir dist/build/hot/hot-tmp -i -idist/build/hot/hot-tmp -i. -idist/build/hot/autogen -idist/build/global-autogen -Idist/build/hot/autogen -Idist/build/global-autogen -Idist/build/hot/hot-tmp -optP-include -optPdist/build/hot/autogen/cabal_macros.h -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db dist/package.conf.inplace -package-id aeson-2.1.2.1-MiGB4vVdVTKjawwWpWZak -package-id array-0.5.4.0 -package-id base-4.16.4.0 -package-id binary-0.8.9.0 -package-id binary-conduit-1.3.1-368gbI3Smq2IqDHr7hlC1L -package-id bytestring-0.11.4.0 -package-id conduit-1.3.5-I1yME8FazApCwMxrjlqtlX -package-id conduit-extra-1.3.6-7WSkrClzZFICRUFqoKKIl3 -package-id errors-2.3.0-Jzdy0VKBDKeCT1xBIdKXUY -package-id hOpenPGP-2.9.8-DcZORBzTPMj4n8tJvl7hE6 -package-id lens-5.2.3-yh7wRKgLG61mi488cKbIH -package-id monad-loops-0.4.3-1PoLx7GFY2f6JjKzBe0o6E -package-id openpgp-asciiarmor-0.1.2-LrUOJzps277LO7FoIL5Rut -package-id optparse-applicative-0.17.0.0-4t835L1BgV19moSYjhXGkU -package-id prettyprinter-1.7.1-BxCO6Qgp3gl9wR9wXXBLKr -package-id prettyprinter-convert-ansi-wl-pprint-1.1.2-KFVoYDOihQkGDCAVEak7Gk -package-id text-1.2.5.0 -package-id transformers-0.5.6.2 -package-id yaml-0.11.11.2-C2OItjIjAVyAQd4zOGsfpE -XHaskell2010 ./hot.hs HOpenPGP.Tools.Common Paths_hopenpgp_tools HOpenPGP.Tools.Armor HOpenPGP.Tools.Lexer HOpenPGP.Tools.Parser -Wall
[1 of 6] Compiling HOpenPGP.Tools.Armor ( HOpenPGP/Tools/Armor.hs, dist/build/hot/hot-tmp/HOpenPGP/Tools/Armor.dyn_o )
[...]
Linking dist/build/hot/hot ...
creating dist/build/hokey
creating dist/build/hokey/autogen
creating dist/build/hokey/autogen
Preprocessing executable 'hokey' for hopenpgp-tools-0.23.7..
Building executable 'hokey' for hopenpgp-tools-0.23.7..
creating dist/build/hokey
creating dist/build/hokey/hokey-tmp
Environment: [("BUILDTOOL","devtools"),("BUILDTOOLVER","1:1.0.4-1-any"),("CFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection"),("CHOST","x86_64-pc-linux-gnu"),("COMMAND_MODE","legacy"),("CXXFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS"),("DEBUGINFOD_URLS","https://debuginfod.archlinux.org "),("HOME","/build"),("LANG","C.UTF-8"),("LDFLAGS","-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"),("LOGNAME","builduser"),("MAIL","/var/mail/builduser"),("MAKEFLAGS","-j17"),("OLDPWD","/build/hopenpgp-tools/src"),("PATH","/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl"),("PWD","/build/hopenpgp-tools/src/hopenpgp-tools-0.23.7"),("PYTHONHASHSEED","0"),("SHELL","/bin/bash"),("SHLVL","1"),("SOURCE_DATE_EPOCH","1697666523"),("SUDO_COMMAND","/bin/bash -c bash -c cd\\ /startdir;\\ makepkg\\ \"$@\" -bash --syncdeps --noconfirm --log --holdver --skipinteg"),("SUDO_GID","0"),("SUDO_UID","0"),("SUDO_USER","root"),("TERM","xterm-256color"),("TEXTDOMAIN","pacman-scripts"),("TEXTDOMAINDIR","/usr/share/locale"),("USER","builduser")]
/usr/bin/ghc --make -no-link -fbuilding-cabal-package -O -j17 -dynamic -fPIC -osuf dyn_o -hisuf dyn_hi -outputdir dist/build/hokey/hokey-tmp -odir dist/build/hokey/hokey-tmp -hidir dist/build/hokey/hokey-tmp -stubdir dist/build/hokey/hokey-tmp -i -idist/build/hokey/hokey-tmp -i. -idist/build/hokey/autogen -idist/build/global-autogen -Idist/build/hokey/autogen -Idist/build/global-autogen -Idist/build/hokey/hokey-tmp -optP-include -optPdist/build/hokey/autogen/cabal_macros.h -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db dist/package.conf.inplace -package-id aeson-2.1.2.1-MiGB4vVdVTKjawwWpWZak -package-id base-4.16.4.0 -package-id base16-bytestring-1.0.2.0-ItcCu34qlNL70gvStrJ4cb -package-id binary-0.8.9.0 -package-id binary-conduit-1.3.1-368gbI3Smq2IqDHr7hlC1L -package-id bytestring-0.11.4.0 -package-id conduit-1.3.5-I1yME8FazApCwMxrjlqtlX -package-id conduit-extra-1.3.6-7WSkrClzZFICRUFqoKKIl3 -package-id containers-0.6.5.1 -package-id cryptonite-0.30-7MiBfVxmkkuHSK74fv9aIk -package-id errors-2.3.0-Jzdy0VKBDKeCT1xBIdKXUY -package-id hOpenPGP-2.9.8-DcZORBzTPMj4n8tJvl7hE6 -package-id http-client-0.7.13.1-JyGR7HnOnawGKIkyoyWU8v -package-id http-client-tls-0.3.6.3-GZ4hD7jm3nzBMVGJZ8AGam -package-id http-types-0.12.3-27zryvtbXffG9f2LeXjTHl -package-id lens-5.2.3-yh7wRKgLG61mi488cKbIH -package-id memory-0.17.0-IlQuKU41I5I99JudO5chqg -package-id openpgp-asciiarmor-0.1.2-LrUOJzps277LO7FoIL5Rut -package-id optparse-applicative-0.17.0.0-4t835L1BgV19moSYjhXGkU -package-id prettyprinter-1.7.1-BxCO6Qgp3gl9wR9wXXBLKr -package-id prettyprinter-ansi-terminal-1.1.3-DwJyDWxSVLFLNNM4VKftV7 -package-id prettyprinter-convert-ansi-wl-pprint-1.1.2-KFVoYDOihQkGDCAVEak7Gk -package-id text-1.2.5.0 -package-id time-1.11.1.1 -package-id time-locale-compat-0.1.1.5-LoE3V3We7KBAnFwKjT98C2 -package-id transformers-0.5.6.2 -package-id yaml-0.11.11.2-C2OItjIjAVyAQd4zOGsfpE -XHaskell2010 ./hokey.hs HOpenPGP.Tools.Common Paths_hopenpgp_tools HOpenPGP.Tools.HKP HOpenPGP.Tools.TKUtils -Wall
[1 of 5] Compiling HOpenPGP.Tools.TKUtils ( HOpenPGP/Tools/TKUtils.hs, dist/build/hokey/hokey-tmp/HOpenPGP/Tools/TKUtils.dyn_o )
[...]
Linking...
Environment: [("BUILDTOOL","devtools"),("BUILDTOOLVER","1:1.0.4-1-any"),("CFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection"),("CHOST","x86_64-pc-linux-gnu"),("COMMAND_MODE","legacy"),("CXXFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS"),("DEBUGINFOD_URLS","https://debuginfod.archlinux.org "),("HOME","/build"),("LANG","C.UTF-8"),("LDFLAGS","-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"),("LOGNAME","builduser"),("MAIL","/var/mail/builduser"),("MAKEFLAGS","-j17"),("OLDPWD","/build/hopenpgp-tools/src"),("PATH","/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl"),("PWD","/build/hopenpgp-tools/src/hopenpgp-tools-0.23.7"),("PYTHONHASHSEED","0"),("SHELL","/bin/bash"),("SHLVL","1"),("SOURCE_DATE_EPOCH","1697666523"),("SUDO_COMMAND","/bin/bash -c bash -c cd\\ /startdir;\\ makepkg\\ \"$@\" -bash --syncdeps --noconfirm --log --holdver --skipinteg"),("SUDO_GID","0"),("SUDO_UID","0"),("SUDO_USER","root"),("TERM","xterm-256color"),("TEXTDOMAIN","pacman-scripts"),("TEXTDOMAINDIR","/usr/share/locale"),("USER","builduser")]
/usr/bin/ghc --make -fbuilding-cabal-package -O -dynamic -fPIC -osuf dyn_o -hisuf dyn_hi -outputdir dist/build/hokey/hokey-tmp -odir dist/build/hokey/hokey-tmp -hidir dist/build/hokey/hokey-tmp -stubdir dist/build/hokey/hokey-tmp -i -idist/build/hokey/hokey-tmp -i. -idist/build/hokey/autogen -idist/build/global-autogen -Idist/build/hokey/autogen -Idist/build/global-autogen -Idist/build/hokey/hokey-tmp -optP-include -optPdist/build/hokey/autogen/cabal_macros.h '-dynload deploy' -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/array-0.5.4.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/base-4.16.4.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/binary-0.8.9.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/bytestring-0.11.4.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/containers-0.6.5.1 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/deepseq-1.4.6.1 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/directory-1.3.6.2 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/exceptions-0.10.4 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/filepath-1.4.2.2 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/ghc-bignum-1.2 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/ghc-boot-th-9.2.8 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/ghc-prim-0.8.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/integer-gmp-1.1 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/mtl-2.2.2 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/parsec-3.1.15.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/pretty-1.1.3.6 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/process-1.6.16.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/rts -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/stm-2.5.0.2 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/template-haskell-2.18.0.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/text-1.2.5.0 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/time-1.11.1.1 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/transformers-0.5.6.2 -optl-Wl,-rpath,/usr/lib/ghc-9.2.8/unix-2.7.2.2 -optl-Wl,-rpath,/usr/lib -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db dist/package.conf.inplace -package-id aeson-2.1.2.1-MiGB4vVdVTKjawwWpWZak -package-id base-4.16.4.0 -package-id base16-bytestring-1.0.2.0-ItcCu34qlNL70gvStrJ4cb -package-id binary-0.8.9.0 -package-id binary-conduit-1.3.1-368gbI3Smq2IqDHr7hlC1L -package-id bytestring-0.11.4.0 -package-id conduit-1.3.5-I1yME8FazApCwMxrjlqtlX -package-id conduit-extra-1.3.6-7WSkrClzZFICRUFqoKKIl3 -package-id containers-0.6.5.1 -package-id cryptonite-0.30-7MiBfVxmkkuHSK74fv9aIk -package-id errors-2.3.0-Jzdy0VKBDKeCT1xBIdKXUY -package-id hOpenPGP-2.9.8-DcZORBzTPMj4n8tJvl7hE6 -package-id http-client-0.7.13.1-JyGR7HnOnawGKIkyoyWU8v -package-id http-client-tls-0.3.6.3-GZ4hD7jm3nzBMVGJZ8AGam -package-id http-types-0.12.3-27zryvtbXffG9f2LeXjTHl -package-id lens-5.2.3-yh7wRKgLG61mi488cKbIH -package-id memory-0.17.0-IlQuKU41I5I99JudO5chqg -package-id openpgp-asciiarmor-0.1.2-LrUOJzps277LO7FoIL5Rut -package-id optparse-applicative-0.17.0.0-4t835L1BgV19moSYjhXGkU -package-id prettyprinter-1.7.1-BxCO6Qgp3gl9wR9wXXBLKr -package-id prettyprinter-ansi-terminal-1.1.3-DwJyDWxSVLFLNNM4VKftV7 -package-id prettyprinter-convert-ansi-wl-pprint-1.1.2-KFVoYDOihQkGDCAVEak7Gk -package-id text-1.2.5.0 -package-id time-1.11.1.1 -package-id time-locale-compat-0.1.1.5-LoE3V3We7KBAnFwKjT98C2 -package-id transformers-0.5.6.2 -package-id yaml-0.11.11.2-C2OItjIjAVyAQd4zOGsfpE -XHaskell2010 ./hokey.hs HOpenPGP.Tools.Common Paths_hopenpgp_tools HOpenPGP.Tools.HKP HOpenPGP.Tools.TKUtils -o dist/build/hokey/hokey -Wall
Linking dist/build/hokey/hokey ...
creating dist/build/hkt
creating dist/build/hkt/autogen
creating dist/build/hkt/autogen
Preprocessing executable 'hkt' for hopenpgp-tools-0.23.7..
Building executable 'hkt' for hopenpgp-tools-0.23.7..
creating dist/build/hkt
creating dist/build/hkt/hkt-tmp
Environment: [("BUILDTOOL","devtools"),("BUILDTOOLVER","1:1.0.4-1-any"),("CFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection"),("CHOST","x86_64-pc-linux-gnu"),("COMMAND_MODE","legacy"),("CXXFLAGS","-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS"),("DEBUGINFOD_URLS","https://debuginfod.archlinux.org "),("HOME","/build"),("LANG","C.UTF-8"),("LDFLAGS","-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"),("LOGNAME","builduser"),("MAIL","/var/mail/builduser"),("MAKEFLAGS","-j17"),("OLDPWD","/build/hopenpgp-tools/src"),("PATH","/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl"),("PWD","/build/hopenpgp-tools/src/hopenpgp-tools-0.23.7"),("PYTHONHASHSEED","0"),("SHELL","/bin/bash"),("SHLVL","1"),("SOURCE_DATE_EPOCH","1697666523"),("SUDO_COMMAND","/bin/bash -c bash -c cd\\ /startdir;\\ makepkg\\ \"$@\" -bash --syncdeps --noconfirm --log --holdver --skipinteg"),("SUDO_GID","0"),("SUDO_UID","0"),("SUDO_USER","root"),("TERM","xterm-256color"),("TEXTDOMAIN","pacman-scripts"),("TEXTDOMAINDIR","/usr/share/locale"),("USER","builduser")]
/usr/bin/ghc --make -no-link -fbuilding-cabal-package -O -j17 -dynamic -fPIC -osuf dyn_o -hisuf dyn_hi -outputdir dist/build/hkt/hkt-tmp -odir dist/build/hkt/hkt-tmp -hidir dist/build/hkt/hkt-tmp -stubdir dist/build/hkt/hkt-tmp -i -idist/build/hkt/hkt-tmp -i. -idist/build/hkt/autogen -idist/build/global-autogen -Idist/build/hkt/autogen -Idist/build/global-autogen -Idist/build/hkt/hkt-tmp -optP-include -optPdist/build/hkt/autogen/cabal_macros.h -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db dist/package.conf.inplace -package-id aeson-2.1.2.1-MiGB4vVdVTKjawwWpWZak -package-id array-0.5.4.0 -package-id base-4.16.4.0 -package-id binary-0.8.9.0 -package-id binary-conduit-1.3.1-368gbI3Smq2IqDHr7hlC1L -package-id bytestring-0.11.4.0 -package-id conduit-1.3.5-I1yME8FazApCwMxrjlqtlX -package-id conduit-extra-1.3.6-7WSkrClzZFICRUFqoKKIl3 -package-id containers-0.6.5.1 -package-id directory-1.3.6.2 -package-id errors-2.3.0-Jzdy0VKBDKeCT1xBIdKXUY -package-id fgl-5.8.1.1-IPRi1EtwRJbDQEGxxbgmDj -package-id graphviz-2999.20.1.0-AhVquVe4TYj3vL1ZKEuPsd -package-id hOpenPGP-2.9.8-DcZORBzTPMj4n8tJvl7hE6 -package-id ixset-typed-0.5.1.0-9K1CbxhMZcAChWrU5OkKVS -package-id lens-5.2.3-yh7wRKgLG61mi488cKbIH -package-id monad-loops-0.4.3-1PoLx7GFY2f6JjKzBe0o6E -package-id optparse-applicative-0.17.0.0-4t835L1BgV19moSYjhXGkU -package-id prettyprinter-1.7.1-BxCO6Qgp3gl9wR9wXXBLKr -package-id prettyprinter-convert-ansi-wl-pprint-1.1.2-KFVoYDOihQkGDCAVEak7Gk -package-id resourcet-1.2.6-G7IPkldIkPEAy1hq8sIYB5 -package-id text-1.2.5.0 -package-id time-1.11.1.1 -package-id transformers-0.5.6.2 -package-id unordered-containers-0.2.19.1-4IM4SZMHfKhJeCsvGt1dd5 -package-id yaml-0.11.11.2-C2OItjIjAVyAQd4zOGsfpE -XHaskell2010 ./hkt.hs HOpenPGP.Tools.Common Paths_hopenpgp_tools HOpenPGP.Tools.Lexer HOpenPGP.Tools.Parser -Wall
[1 of 5] Compiling HOpenPGP.Tools.Lexer ( dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs, dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.dyn_o )
[...]

Of note is the line

/usr/bin/alex -g -o dist/build/hot/hot-tmp/HOpenPGP/Tools/Lexer.hs HOpenPGP/Tools/Lexer.x

which only occurs once and is seemingly only run for hot, and the entry in Environment

("PATH","/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl")

which is repeated several times, but always the same.

I've also extracted the dist/build/hot/hot-tmp/HOpenPGP/Tools/Lexer.hs which is mentioned above as the target of the alex command, and the dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs, which is the Lexer.hs for hkt.

For completeness, here is the hopenpgp-tools.cabal for 0.23.7, as available on Hackage.

runhaskell Setup

runhaskell Setup is a low-level interface, whereas cabal-install provides a high-level interface, which is reflected in the many dependencies it draws upon. The difference starts to matter when considering that cabal build does a lot more than just building a package, not least including its own system for package management.

andreasabel commented 1 year ago

I see that Lexer.hs and Parser.hs (dating June 2022) are included in the tarball for 0.23.7:

$ cabal get hopenpgp-tools-0.23.7
Unpacking to hopenpgp-tools-0.23.7/

$ find . -name "Lexer.hs"
./hopenpgp-tools-0.23.7/dist/build/hop/hop-tmp/HOpenPGP/Tools/Lexer.hs
./hopenpgp-tools-0.23.7/dist/build/hot/hot-tmp/HOpenPGP/Tools/Lexer.hs
./hopenpgp-tools-0.23.7/dist/build/hkt/hkt-tmp/HOpenPGP/Tools/Lexer.hs

This may prevent their (re-)generation with alex in install methods like runhaskell Setup build. This explains the build failure with recent GHCs.

Apparently, their tarball was not created with cabal sdist, because that does not package the generated lexer and parser. So, it could be a packaging error on the side of the hopenpgp-tools. (I have made a similar error in the past, when publishing https://hackage.haskell.org/package/BNFC-2.8.3.)

andreasabel commented 1 year ago

there is nothing to suggest that 0.23.8 fixes this issue,

In fact, I checked the tarball of 0.23.8, and it no longer includes the generated Lexer.hs. So I think it would fix the problem indeed.

@Vekhir : Could you report the progress to https://salsa.debian.org/clint/hopenpgp-tools/-/issues/6 ? I don't have an account there and cannot log in with my github account there.

Vekhir commented 1 year ago

Adding a step to first delete the dist/ directory before building did indeed fix the issue.

there is nothing to suggest that 0.23.8 fixes this issue,

In fact, I checked the tarball of 0.23.8, and it no longer includes the generated Lexer.hs. So I think it would fix the problem indeed.

One takeaway for me is to always check out the updates to get the differential, even if it introduces new variables. Though I will take the fix over updating in this case, for now at least.

Could you report the progress to https://salsa.debian.org/clint/hopenpgp-tools/-/issues/6 ?

Done!

Thanks for your help in investigating the issue. Can you close this issue as completed (vs. not planned)?

andreasabel commented 1 year ago

@Vekhir: I am happy that I could help you solve this issue!

I suppose "closing as not planned" does include the case that the issue was never on our side (alex) but caused by a third party (here: hopenpgp-tools).

Would you please reopen https://salsa.debian.org/clint/hopenpgp-tools/-/issues/6 ? I think the maintainer should deprecate the affected versions on hackage, and ideally also make them unbuildable. This can be done by Hackage revisions by the maintainer.

Vekhir commented 1 year ago

the maintainer should deprecate the affected versions on hackage

What are the reasons for that? There were no issues with cabal build, and for older versions of GHC (< 9.2) they work as is. The public API of hopenpgp-tools isn't the issue aswell. So this seems quite drastic to me.

andreasabel commented 1 year ago

The problem is that the wrong packaging is a mine that people may randomly step on (like it happened for you). There should be a clear indication of this issue. Deprecation would be a clear marking. (Note that this would still people to install this version if they insist, only it would by default not chosen by cabal in constraint solving.) And a bound base < 4.16 would indicate that there are (could be) build problems with GHC 9.2 and above. Of course, I assume here that the older versions are not essential to anyone. Usually folks are fine to use the newest version.

haskell / alex