interface-file errors when using Template Haskell since ghcide 0.2.0

[maralorn]

I have a project that compiles with cabal and often works without a problem with ghcide. But sometimes I see issues like this:

 [typecheck] [E] • Can't find interface-file declaration for data constructor SortPosition
    Probable cause: bug in .hi-boot file, or inconsistent .hi file
    Use -ddump-if-trace to get an idea of which file caused the error

I have seen this at different positions in my code, but sadly, also because of the heisenbugishness of this bug, I couldn‘t construct a minimal failing example yet.

This issue didn‘t happen on ghcide 0.1.0.

On thing of note: I have only met this issue on data constructors that where called like the Type they inhabit. Renaming the Type fixed the problem for me. But that might just have been luck.

EDIT: The Type is generated via Template Haskell.

[pepeiborra]

Thanks for the bug report. It would certainly be helpful to have a minimal repro case, as well as details of your setup, including:

• cradle
• GHC version
• Editor
• OS

[maralorn]

I am very sorry, but condensing down to a minimal repro case might take me a while. I just thought I'll put this out here, so that you‘ve heard about this.

I can tell you my setup right away:

• cradle: None configured. It works out of the box without hie.yaml file but I think it uses the cabal cradle, as that is how I configure my project. Also the issue remained with an explicit cabal cradle configured.

• ghcide version: 0.2.0 (GHC: 8.6.5) (PATH: /nix/store/pc7y00g83kaisw2y6psahhhd3438xjaa-ghcide-0.2.0/bin/ghcide)

• neovim + coc.nvim

• nixos (This is running in a nix-shell based on reflex-platform, which means it's based on nixpkgs 19.09).

[maralorn]

I have forgotten to mention a very important point. The Type in question is generated via TemplateHaskell. Sorry, about that …

[tomsmeding]

I have a very minimal testcase that produces this kind of error: https://gist.github.com/tomsmeding/326fa7361ee19af34c06cb6f7ad7d20e This project can be successfully built by both Cabal and Stack, but running ghcide fails with the "Can't find interface-file declaration" error.

The problem here is that the TemplateHaskell splice refers to a module-local binding that is not in scope in the module where the TH splice is used. If glorifiedID is added to the export list of Error.hs, ghcide loads the project successfully.

[maoe]

@tomsmeding's reproducer looks identical to the repro in https://github.com/haskell/haskell-language-server/issues/149.

[pepeiborra]

@tomsmeding thanks for the repro. Could you do a little bit further analysis to help us understand the issue?

1. stack/cabal cradle. Is it stack specific or does it repro also with Cabal?
2. ghc 8.8 / 8.10. Is it specific to 8.8 or does it repro also with 8.10?
3. ghcide 0.2.0 / HEAD. What versions have you tested with?

My setup at home with Nix, ghc 8.10, ghcide HEAD and the default cradle (which seems to use stack) does not seem to reproduce the issue.

[pepeiborra]

Good news, I gave it another go and managed to reproduce it both with 8.8 and 8.10.

[pepeiborra]

I think I know what the problem is: when writing interface files for TH modules, type checking is not enough. We must extract the interface file after running the desugarer/simplifier.

Fixing this will take some staring at the ghc-api to figure out how/where to extract the ModIface objects from, followed by a significant refactoring to lift the assumption that interface files are the output of type checking, which is not the case when TH is involved. See also haskell/haskell-language-server#867

[pepeiborra]

Given that none of my personal projects (nor my employer) use template haskell, there's little motivation for me to fix this. It's up for grabs, and I'm happy to provide more directions and review merge requests

[mpickering]

@pepeiborra Why do you need to run the simplifier to fix this problem? TH splices are run during renaming.

[pepeiborra]

I compared the interface files produced by ghcide with the ones produced by ghc, and found that the former lacked e.g. entries for top level names created by TH splices.

But if TH splices are run during renaming there must be something else missing.

[maralorn]

According to @mpickering the problem is in ghc where mkBootModDetailsTc ignores the values from tcg_keep. I will see if I can come up with a Bugfix PR.

[maralorn]

This issue is fixed on the wz1000 hls-3 branch and while therefore reach a hls near you soon.

[pepeiborra]

Awesome!! Please send a PR to ghcide HEAD, so that the fix will reach ghcide users too...

[maralorn]

Sadly I have too admit, that I fixed a bug, but not my bug. So this cannot be closed. I have a test reproducing my bug in this commit: https://github.com/digital-asset/ghcide/commit/985268dec97783b44d0339bf32d0cf084b903410

[maralorn]

Just a quick heads-up as to where we stand. This issue has now served as a discussion for two different bugs leading to very similar error messages.

• The first bug happens when symbols need to be exported because they occur in a TH splice. (or something). That is fixed in haskell/ghcide#741, which just needs a tiny bit of compatibility fixes before it can be merged.

• The second bug occurs in some situations where a data constructor is generated with template haskell. We are right now working on finding a solution for that. The failing test is on the fix-more-th branch (which does not yet contain a fix sadly) on my fork.

The current result of investigation is, that Template Haskell sometimes triggers a retypechecking of a loaded module and when this occurs we observe that the unique id assigned to the data constructor apparently is not stable, which leads to inconsistencies and the error-message.

Example: First typecheck gives value: fake_uid:B.A{d a46h} second typecheck gives: fake_uid:B.A{d a46I}, error message is Can't find interface-file declaration for data constructor fake_uid:B.A{d a46h}

(Everytime I write "We" I mean "I" in the sense of trying to push this forward, but say "We" because I couldn‘t do anything without the ton of help I receive on IRC.)

[wz1000]

Th error happens because the TypeEnv for module B gets into an inconsistent state after two typechecks.

After the first typecheck, we get the following TypeEnv (In the ModDetails of the TypecheckedModule that is generated in typecheckModule):

[a46h :-> Data constructor âfake_uid:B.A{d a46h}â,
 r45p :-> Type constructor âfake_uid:B.A{tc r45p}â,
 r46i :-> Identifier âfake_uid:B.$tc'A{v r46i}â,
 r46t :-> Identifier âfake_uid:B.A{v r46t}â,
 r46u :-> Identifier âfake_uid:B.$tcA{v r46u}â,
 r46w :-> Identifier âfake_uid:B.$trModule{v r46w}â]

After the second typecheck, the TypeEnv becomes:

[a46I :-> Data constructor âfake_uid:B.A{d a46I}â,
 r45p :-> Type constructor âfake_uid:B.A{tc r45p}â,
 r46i :-> Identifier âfake_uid:B.$tc'A{v r46i}â,
 r46t :-> Identifier âfake_uid:B.A{v r46t}â,
 r46u :-> Identifier âfake_uid:B.$tcA{v r46u}â,
 r46w :-> Identifier âfake_uid:B.$trModule{v r46w}â]

Finally, the TypeEnv that is found in the ModDetails generated in loadDepModuleIO (and thus the one stored in the HPT) is a mangled version of the previous two:

[a46I :-> Data constructor âfake_uid:B.A{d a46h}â,
 r45p :-> Type constructor âfake_uid:B.A{tc r45p}â,
 r46i :-> Identifier âfake_uid:B.$tc'A{v r46i}â,
 r46t :-> Identifier âfake_uid:B.A{v r46t}â,
 r46u :-> Identifier âfake_uid:B.$tcA{v r46u}â,
 r46w :-> Identifier âfake_uid:B.$trModule{v r46w}â]

This failure only happens when the name for the data constructor A is generated via newName. If we use mkName instead, all the occurrence of A{d} get the same unique, and so the final TypeEnv is in a consistent state.

The failure happens when GHC fails to look up the unique a46h for the datacon A in the final TypeEnv

[wz1000]

I can confirm that the testcase works if we only typecheck the file once.

Perhaps its time to revive https://github.com/mpickering/ghcide/pull/43