Conditionally built module are not loaded if Cabal disables them and stack.yaml reenables them with stack (revisit after commercialhaskell/stack#5480)

konn commented 3 years ago

Summary

If some modules are conditionally built depending on flags, and the flag is off by default, then even if one explicitly enable that flag in corresponding stack.yaml used by HLS, those modules are not loaded automatically.

Your environment

Output of haskell-language-server --probe-tools or haskell-language-server-wrapper --probe-tools:

$ ~/Library/Application\ Support/Code/User/globalStorage/haskell.haskell/haskell-language-server-wrapper-0.8.0-darwin --probe-tools
haskell-language-server version: 0.8.0.0 (GHC: 8.10.1) (PATH: /Users/hiromi/Library/Application Support/Code/User/globalStorage/haskell.haskell/haskell-language-server-wrapper-0.8.0-darwin) (GIT hash: eb58f13f7b8e4f9bc771af30ff9fd82dc4309ff5)
Tool versions found on the $PATH
cabal:          3.2.0.0
stack:          2.5.1
ghc:            8.6.5

Which lsp-client do you use: VSCode

Describe your project (alternative: link to the project): I have the repro repo for this.

The project has the following module hierarchy:

src
├── Lib
│   └── Internal.hs
├── Lib.hs
├── Lib2
│   └── Internal.hs
└── Lib2.hs

Here, Lib imports Lib.Internal and Lib2 imports Lib2.Internal. Lib2 and Lib2.Internal are conditionally built based on the flag dsiable-lib2, which is on by default in cabal file. In stack.yaml, which is used for both building and HLS, disable-lib2 flag is explicitly disabled as follows, hence Lib2* must be built with stack:

flags:
  hls-conditional-modules:
    disable-lib2: false

Contents of hie.yaml: Not needed; but I have one with generated by gen-hie:

cradle:
  stack:
    - path: "src"
      component: "hls-conditional-modules:lib"

Steps to reproduce

Open project directory, without any file open.
Open src/Lib2.hs
Wait for modules to be loaded

Expected behaviour

Lib2 will be successfully loaded without any error.

Actual behaviour

HLS could not load Lib2.Internal:

{
    "resource": "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/src/Lib2.hs",
    "owner": "Haskell (hls-conditional-modules)",
    "severity": 8,
    "message": "Could not load module ‘Lib2.Internal’\nIt is a member of the hidden package ‘hls-conditional-modules-0.1.0.0’.\nYou can run ‘:set -package hls-conditional-modules’ to expose it.\n(Note: this unloads all the modules in the current scope.)",
    "source": "not found",
    "startLineNumber": 3,
    "startColumn": 8,
    "endLineNumber": 3,
    "endColumn": 21
}

If one opens src/Lib2/Internal.hs, the above error dissapears. Additionally, If one disables disable-lib2 flag in Cabal file, Lib2.Internal will be loaded just by opening Lib2.hs alone.

Include debug information

Execute in the root of your project the command haskell-language-server --debug . and paste the logs here:

Debug output:

``` $ ~/Library/Application\ Support/Code/User/globalStorage/haskell.haskell/haskell-language-server-wrapper-0.8.0-darwin --debug Found "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/hie.yaml" for "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/a" Module "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/a" is loaded by Cradle: Cradle {cradleRootDir = "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules", cradleOptsProg = CradleAction: Stack} Run entered for haskell-language-server-wrapper(haskell-language-server-wrapper-0.8.0-darwin) Version 0.8.0.0, Git revision eb58f13f7b8e4f9bc771af30ff9fd82dc4309ff5 (dirty) x86_64 ghc-8.10.1 Current directory: /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules Operating system: darwin Arguments: ["--debug"] Cradle directory: /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules Cradle type: Stack Tool versions found on the $PATH cabal: 3.2.0.0 stack: 2.5.1 ghc: 8.6.5 Consulting the cradle to get project GHC version... Project GHC version: 8.10.3 haskell-language-server exe candidates: ["haskell-language-server-8.10.3","haskell-language-server-8.10","haskell-language-server"] Cannot find any haskell-language-server exe, looked for: haskell-language-server-8.10.3, haskell-language-server-8.10, haskell-language-server 金 01 29 hiromi@Swing ~/Documents/Programming/Haskell/git/hls-conditional-modules $ ~/Library/Application\ Support/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.8.0-darwin-8.10.3 --debug haskell-language-server version: 0.8.0.0 (GHC: 8.10.3) (PATH: /Users/hiromi/Library/Application Support/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.8.0-darwin-8.10.3) (GIT hash: eb58f13f7b8e4f9bc771af30ff9fd82dc4309ff5) (haskell-language-server)Ghcide setup tester in /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules. Report bugs at https://github.com/haskell/haskell-language-server/issues Tool versions found on the $PATH cabal: 3.2.0.0 stack: 2.5.1 ghc: 8.6.5 Step 1/4: Finding files to test in /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules Found 5 files Step 2/4: Looking for hie.yaml files that control setup Found 1 cradle Step 3/4: Initializing the IDE Step 4/4: Type checking the files [INFO] Consulting the cradle for "src/Lib/Internal.hs" Output from setting up the cradle Cradle {cradleRootDir = "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules", cradleOptsProg = CradleAction: Stack} > hls-conditional-modules> configure (lib) > Configuring hls-conditional-modules-0.1.0.0... > hls-conditional-modules> initial-build-steps (lib) > Configuring GHCi with the following packages: hls-conditional-modules > /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/.stack-work/install/x86_64-osx/9a76837884c2ac46e9c194c478c3b6e3b88485dcc802a09c5f1784038d24c283/8.10.3/pkgdb:/Users/hiromi/.stack/snapshots/x86_64-osx/9a76837884c2ac46e9c194c478c3b6e3b88485dcc802a09c5f1784038d24c283/8.10.3/pkgdb:/Users/hiromi/.stack/programs/x86_64-osx/ghc-8.10.3/lib/ghc-8.10.3/package.conf.d [INFO] Using interface files cache dir: ghcide [INFO] Making new HscEnv[main] [INFO] Consulting the cradle for "src/Lib2.hs" Output from setting up the cradle Cradle {cradleRootDir = "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules", cradleOptsProg = CradleAction: Stack} > hls-conditional-modules> configure (lib) > Configuring hls-conditional-modules-0.1.0.0... > hls-conditional-modules> initial-build-steps (lib) > Configuring GHCi with the following packages: hls-conditional-modules > /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/.stack-work/install/x86_64-osx/9a76837884c2ac46e9c194c478c3b6e3b88485dcc802a09c5f1784038d24c283/8.10.3/pkgdb:/Users/hiromi/.stack/snapshots/x86_64-osx/9a76837884c2ac46e9c194c478c3b6e3b88485dcc802a09c5f1784038d24c283/8.10.3/pkgdb:/Users/hiromi/.stack/programs/x86_64-osx/ghc-8.10.3/lib/ghc-8.10.3/package.conf.d [INFO] Using interface files cache dir: ghcide [INFO] Using interface files cache dir: ghcide [INFO] Making new HscEnv[main,main] [INFO] Consulting the cradle for "src/Lib2/Internal.hs" Output from setting up the cradle Cradle {cradleRootDir = "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules", cradleOptsProg = CradleAction: Stack} > hls-conditional-modules> configure (lib) > Configuring hls-conditional-modules-0.1.0.0... > hls-conditional-modules> initial-build-steps (lib) > Configuring GHCi with the following packages: hls-conditional-modules > /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/.stack-work/install/x86_64-osx/9a76837884c2ac46e9c194c478c3b6e3b88485dcc802a09c5f1784038d24c283/8.10.3/pkgdb:/Users/hiromi/.stack/snapshots/x86_64-osx/9a76837884c2ac46e9c194c478c3b6e3b88485dcc802a09c5f1784038d24c283/8.10.3/pkgdb:/Users/hiromi/.stack/programs/x86_64-osx/ghc-8.10.3/lib/ghc-8.10.3/package.conf.d [INFO] Using interface files cache dir: ghcide [INFO] Using interface files cache dir: ghcide [INFO] Using interface files cache dir: ghcide [INFO] Making new HscEnv[main,main,main] [INFO] Consulting the cradle for "Setup.hs" Output from setting up the cradle Cradle {cradleRootDir = "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules", cradleOptsProg = CradleAction: Stack} File: /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/Setup.hs Hidden: no Range: 1:1-2:1 Source: cradle Severity: DsError Message: Multi Cradle: No prefixes matched pwd: /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules filepath: /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/Setup.hs prefixes: ("src",Stack {component = Just "hls-conditional-modules:lib", stackYaml = Nothing}) Files that failed: [INFO] finish: User TypeCheck (took 0.06s) * /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/Setup.hs ```

Paste the logs from the lsp-client, e.g. for VS Code

LSP logs:

``` [client] run command: "/Users/hiromi/Library/Application Support/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.8.0-darwin-8.10.3 --lsp" [client] debug command: "/Users/hiromi/Library/Application Support/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.8.0-darwin-8.10.3 --lsp" [client] server cwd: undefined haskell-language-server version: 0.8.0.0 (GHC: 8.10.3) (PATH: /Users/hiromi/Library/Application Support/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.8.0-darwin-8.10.3) (GIT hash: eb58f13f7b8e4f9bc771af30ff9fd82dc4309ff5) Starting (haskell-language-server)LSP server... with arguments: LspArguments {argLSP = True, argsCwd = Nothing, argFiles = [], argsShakeProfiling = Nothing, argsTesting = False, argsExamplePlugin = False, argsDebugOn = False, argsLogFile = Nothing, argsThreads = 0, argsProjectGhcVersion = False} with plugins: [PluginId "brittany",PluginId "class",PluginId "eval",PluginId "floskell",PluginId "fourmolu",PluginId "ghcide",PluginId "hlint",PluginId "importLens",PluginId "moduleName",PluginId "ormolu",PluginId "pragmas",PluginId "retrie",PluginId "stylish-haskell",PluginId "tactic"] in directory: /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules If you are seeing this in a terminal, you probably should have run ghcide WITHOUT the --lsp option! Started LSP server in 0.00s 2021-01-29 11:52:18.469968 [ThreadId 25] - Registering ide configuration: IdeConfiguration {workspaceFolders = fromList [NormalizedUri 326243741438682258 "file:///Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules"], clientSettings = hashed Nothing} 2021-01-29 11:52:18.471264 [ThreadId 25] - Configuration changed: Object (fromList [("haskell",Object (fromList [("hlint",Object (fromList [("logLevel",String "info"),("executablePath",String "/Users/hiromi/.local/bin/hlint")])),("logFile",String ""),("updateBehavior",String "keep-up-to-date"),("hlintOn",Bool True),("formatOnImportOn",Bool True),("indentationRules",Object (fromList [("enabled",Bool True)])),("liquidOn",Bool False),("languageServerVariant",String "haskell-language-server"),("serverExecutablePath",String ""),("diagnosticsOnChange",Bool True),("completionSnippetsOn",Bool True),("maxNumberOfProblems",Number 100.0),("formattingProvider",String "fourmolu"),("trace",Object (fromList [("server",String "off")]))]))]) 2021-01-29 11:52:18.471905 [ThreadId 25] - Opened text document: file:///Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/src/Lib2.hs 2021-01-29 11:52:18.472613 [ThreadId 105] - hlint:getIdeas:file:NormalizedFilePath "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/src/Lib2.hs" 2021-01-29 11:52:18.47305 [ThreadId 109] - Consulting the cradle for "src/Lib2.hs" Output from setting up the cradle Cradle {cradleRootDir = "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules", cradleOptsProg = CradleAction: Stack} 2021-01-29 11:52:18.730881 [ThreadId 127] - Plugin.makeCodeLens (ideLogger) > hls-conditional-modules> configure (lib) > Configuring hls-conditional-modules-0.1.0.0... > hls-conditional-modules> initial-build-steps (lib) > Configuring GHCi with the following packages: hls-conditional-modules > /Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/.stack-work/install/x86_64-osx/9a76837884c2ac46e9c194c478c3b6e3b88485dcc802a09c5f1784038d24c283/8.10.3/pkgdb:/Users/hiromi/.stack/snapshots/x86_64-osx/9a76837884c2ac46e9c194c478c3b6e3b88485dcc802a09c5f1784038d24c283/8.10.3/pkgdb:/Users/hiromi/.stack/programs/x86_64-osx/ghc-8.10.3/lib/ghc-8.10.3/package.conf.d 2021-01-29 11:52:22.260249 [ThreadId 109] - Using interface files cache dir: ghcide 2021-01-29 11:52:22.260319 [ThreadId 109] - Making new HscEnv[main] 2021-01-29 11:52:22.273968 [ThreadId 199] - hlint:getIdeas:file:NormalizedFilePath "/Users/hiromi/Documents/Programming/Haskell/git/hls-conditional-modules/src/Lib2.hs" 2021-01-29 11:52:22.27579 [ThreadId 254] - finish: getSession (took 0.00s) 2021-01-29 11:52:22.278784 [ThreadId 350] - finish: codeLens (took 0.00s) 2021-01-29 11:52:22.27927 [ThreadId 364] - finish: (took 0.00s) 2021-01-29 11:52:22.279577 [ThreadId 367] - finish: ModuleName.ghcSession (took 0.00s) 2021-01-29 11:52:22.280069 [ThreadId 372] - finish: ModuleName.GetParsedModule (took 0.00s) 2021-01-29 11:52:57.777813 [ThreadId 433] - finish: CodeAction (took 0.00s) 2021-01-29 11:52:58.002377 [ThreadId 439] - finish: CodeAction:PackageExports (took 0.22s) 2021-01-29 11:52:58.002629 [ThreadId 440] - finish: importLens (took 0.00s) 2021-01-29 11:52:58.002753 [ThreadId 442] - finish: MinimalImports (took 0.00s) 2021-01-29 11:52:58.003021 [ThreadId 444] - finish: addPragma (took 0.00s) 2021-01-29 11:52:58.003235 [ThreadId 446] - finish: retrie (took 0.00s) 2021-01-29 11:52:58.003459 [ThreadId 448] - finish: tactic (took 0.00s) 2021-01-29 11:52:58.564398 [ThreadId 453] - Plugin.makeCodeLens (ideLogger) 2021-01-29 11:52:58.564829 [ThreadId 454] - finish: getSession (took 0.00s) 2021-01-29 11:52:58.565176 [ThreadId 456] - finish: codeLens (took 0.00s) 2021-01-29 11:52:58.565468 [ThreadId 458] - finish: (took 0.00s) 2021-01-29 11:52:58.565849 [ThreadId 460] - finish: ModuleName.ghcSession (took 0.00s) 2021-01-29 11:52:58.566617 [ThreadId 462] - finish: ModuleName.GetParsedModule (took 0.00s) 2021-01-29 11:56:42.555448 [ThreadId 468] - finish: CodeAction (took 0.00s) 2021-01-29 11:56:42.555638 [ThreadId 470] - finish: CodeAction:PackageExports (took 0.00s) 2021-01-29 11:56:42.555941 [ThreadId 472] - finish: importLens (took 0.00s) 2021-01-29 11:56:42.556063 [ThreadId 474] - finish: MinimalImports (took 0.00s) 2021-01-29 11:56:42.55655 [ThreadId 476] - finish: addPragma (took 0.00s) 2021-01-29 11:56:42.556789 [ThreadId 478] - finish: retrie (took 0.00s) 2021-01-29 11:56:42.557005 [ThreadId 480] - finish: tactic (took 0.00s) ```

konn commented 3 years ago

It seems that if we use cabal with cabal.project instead of stack, the problem disappears. This can be checked with cabal branch. And if we disable flag in cabal.project, it reports the hidden-module error as expected.

fendor commented 3 years ago

Looks like a stack bug to me.

I checked locally, stack does not list the targets Lib2 and Lib2.Internal as targets. (As can be checked by invoking hie-bios debug src/Lib2.hs, then looking at the file pointed to by the flag -ghci-script )

Example:

> cat /run/user/1000/haskell-stack-ghci/4a4516ab/ghci-script
:add Lib Lib.Internal
:module + Lib Lib.Internal

You can see that when you execute stack repl, only two modules are imported.

konn commented 3 years ago

So it must be reported to commercialhaskell/stack as an upstream bug, right?

fendor commented 3 years ago

Yeah, ideally

konn commented 3 years ago

OK, I've just reported it as commercialhaskell/stack#5480.

haskell / haskell-language-server