Build instructions in contributing.rst are wrong

[andreasabel]

https://github.com/haskell/happy/blob/535ce96533fb693a8a4441b0ac17cdb78bddfeec/doc/contributing.rst?plain=1#L36-L43

On a fresh working copy of master ( 535ce96533fb693a8a4441b0ac17cdb78bddfeec ):

$ happy -V
Happy Version 1.20.1.1  ...

$ cabal build
....
Preprocessing library for happy-frontend-2.0...
happy: /Users/abel/project/open-source/happy-master/././HappyTemplate-arrays-coerce: openFile: does not exist (No such file or directory)
Error: [Cabal-7125]
Failed to build happy-frontend-2.0 (which is required by exe:happy from happy-2.0).

ATTN: @Ericson2314 @int-index

[int-index]

Sounds like an issue with the happy that you have in your $PATH. Similar issue: https://github.com/haskell/happy/issues/262.

I can't reproduce the failure.

[andreasabel]

Do you mean you can only bootstrap happy-2 by happy-2 ? Then maybe this should be added to the doc.

To me it looks more like happy-2 wants some template that has not been created. Maybe cabal build does not create these templates, or not at the right time?

[int-index]

Do you mean you can only bootstrap happy-2 by happy-2 ?

Not at all. I mean that your installation of happy-1 might be somehow misconfigured.

[andreasabel]

Well, it is not. I use it all the time, to rebuild Agda, to test BNFC etc.

Also, the non-existing file mentioned in the error is in the domain of happy-master:

/Users/abel/project/open-source/happy-master/././HappyTemplate-arrays-coerce

@int-index wrote:

Sounds like an issue with the happy that you have in your $PATH. Similar issue: #262.

You hypothesis there was never confirmed.

And here is the counterevidence: I can exactly reproduce #262 (and my error) in a workflow: https://github.com/haskell/happy/actions/runs/8378168492/job/22942066511?pr=275#step:6:28

happy: /home/runner/work/happy/happy/././HappyTemplate-arrays-coerce: openFile: does not exist (No such file or directory)

The workflow (https://github.com/haskell/happy/commit/c9b18cb7f27e58054e096adc783def2668241776) follows the build instructions in the manual.

Maybe your system isn't clean, that's why you cannot reproduce the issue? You might have happy-2 install artefacts lying around, maybe after running bootstrap.hs or some previous happy-2 installation.

[int-index]

OK, I've reproduced this on another machine. Seems to be a non-deterministic failure, sensitive to build order. Could you try this sequence of commands?

$ cabal build happy-frontend
$ cabal build

Building happy-frontend first seems to help.

[andreasabel]

$ cabal build happy-frontend $ cabal build

Yes this works.

But why? Is the bootstrap turned on just if cabal build happy-frontend is invoked, and not with cabal build all or `cabal build [happy]? https://github.com/haskell/happy/blob/535ce96533fb693a8a4441b0ac17cdb78bddfeec/packages/frontend/happy-frontend.cabal#L41-L46

[int-index]

The bootstrap flag is turned on either way. You can also confirm this by specifying it manually

cabal build happy-frontend -f +bootstrap
cabal build all -f +bootstrap

[int-index]

The failure seems to be caused by cabal extending the environment with happy_backend_glr_datadir, happy_backend_lalr_datadir, happy_datadir, and happy_tabular_datadir.

I can reliably reproduce the failure as follows

happy_datadir="$(pwd)/./." cabal build happy-frontend

[int-index]

Could it be a problem in cabal? Here's the version I have on the machine where the problem is reproducible

$ cabal --version
cabal-install version 3.10.2.1
compiled using version 3.10.1.0 of the Cabal library

And here's the version on the machine where I cannot reproduce the problem

$ cabal --version
cabal-install version 3.10.2.1
compiled using version 3.10.2.1 of the Cabal library

You can see that it uses a newer version of the Cabal library. It also does not extend the environment with happy_datadir.

[int-index]

I can reliably reproduce the failure as follows

No, I take that back. It does reproduce the problem on one machine but not on the other one. I'm lost again.

[sgraf812]

FWIW, I can reproduce with happy_datadir="$(pwd)/./." cabal build happy-frontend and the following cabal:

$ cabal --version
cabal-install version 3.10.2.1
compiled using version 3.10.2.1 of the Cabal library

So it does not appear to be related to the version of Cabal.

On the machine where you can't reproduce, to you happen to have a cabal.project.local file? I currently use cabal configure -f -bootstrap to work around the problem, which creates said file to disable bootstrapping.

[sgraf812]

I could come up with a minimal reproducer for what I think is a bug in Cabal; see https://github.com/haskell/cabal/issues/10072.

[sgraf812]

Given that

1. it is unlikely that the Cabal bug https://github.com/haskell/cabal/issues/10072 gets fixed
2. it would take several years for installed cabal-install binaries to catch up even if that bug gets fixed

I would suggest to simply check in the code of the generated happy parser. It is absolutely rare that we must touch the parser anyway, and it is simple enough to check in the new code anyway. All the dog-fooding arguments still apply and we could reasonably just link .../frontend/.../Bootstrapped.ly into the test folder.

I then wonder what we need the "Oracle" parser for... Would we still need the cabal flag in the first place then?

[int-index]

I then wonder what we need the "Oracle" parser for...

The initial idea (in #170) was to avoid bootstrapping entirely, thus reducing the complexity of the build, and avoiding check-in of generated code. I still think it's a viable route.

But then we decided to make it into an optional flag (in #175), and we actually ended up with more complexity, not less. And it wouldn't be so bad if we didn't start hitting Cabal limitations (#211, #255, #256, https://github.com/haskell/cabal/issues/7189, and now https://github.com/haskell/cabal/issues/10072). And, as far as I can see, there's no one stepping up to address these limitations.

At this point let's just cut our losses and drop the -bootstrap flag and the "oracle" parser with it. They will remain with us in the git history, and we can bring them back if someone, in a feat of enthusiasm, decides to patch Cabal to cooperate.

[int-index]

Here we go: https://github.com/haskell/happy/pull/277

[sgraf812]

Thanks so much, Vlad! Really looking forward to seeing your fix merged.