No hover responses on dhall-haskell in windows

[fendor]

Hi, i've experienced some hangs testing this pr within dhall-haskell project:

• Opening the project with vscode and hie master works (not with all files but it does with ./dhall/src/Dhall.hs)
• When using this branch the load of cradle finishes but hover on any term (or any other action involving hie) is unreponsive: the log is here (including the hover request and the cancellation)
• Testing hie-bios on the project without explicit hie.yaml works: see https://gist.github.com/jneira/65c0ae7359beeb066c0d6174bc700bc1#file-cradle-output

Originally posted by @jneira in https://github.com/haskell/haskell-ide-engine/pull/1126#issuecomment-539237025

cc @jneira

[mpickering]

Did you test this yourself @fendor?

[fendor]

Yes, but I can not reproduce. Cloning dhall-haskell with hie.yaml

cradle: 
  stack:

works for me.

[jneira]

Maybe it is a specific behaviour in windows, i've experienced it in win 7 and 10, with both the hie-bios and the cabal-helper-helper branches, using an explicit cradle like fendor's one and without it .

It works in simple stack projects, though.

[jneira]

I've setup a simple cabal/stack multipackage project: https://github.com/jneira/cabal-multi-test and i've done an extensive testing

TL;DR

• in all succesful cases there is a notificacion in vscode: "lintCmd: no access to the persisted file."
• hover unresponsive only when cradle init doesnt finish

---

• cabal-helper-helper branch with no explicit hie.yaml
  • With stack.yaml and no cabal.project: it chooses the Cabal-Helper-Stack cradle
  • It works setting the test suite name equal in both subpackages, the cradle initialization finishes
  • there is a notificacion in vscode: "lintCmd: no access to the persisted file."
  • Hover responds in all cases
  • Logs: https://gist.github.com/jneira/994301d33c2d28d50a3c5330f44132cb
  • With stack.yaml and cabal.project: it chooses the Cabal-Helper-Cabal-V2 cradle
  • With the same test suite name: the cradle init doesn't finish
    • hover doesn't respond
    • log: https://gist.github.com/jneira/e08161485287c77998b31d9a01dc2ad1
  • With no duplicate test names in both packages the cradle init finishes.
    • there is a notificacion in vscode: "lintCmd: no access to the persisted file."
    • hover responds
    • logs: https://gist.github.com/jneira/b4e6fa867a2e78a1113d3d6cc1e7d56e
• hie-bios branch with no explicit hie.yaml
  • With stack.yaml and cabal.project: it chooses the stack cradle (note that the behaviour is the oppsite of the cabal-helper-helper branch)
  • Hover responds
  • there is a notificacion in vscode: "lintCmd: no access to the persisted file."
  • With no stack.yaml and cabal.project: it chooses the cabal cradle
  • It thows an error (expected?):

    CradleError (ExitFailure 1) "Failed to parse result of calling cabal\ncabal: No targets given and there is no package in the current directory. Use\nthe target 'all' for all packages in the project or specify packages or\ncomponents by name or location. See 'cabal build --help' for more details on\ntarget options.\n\n\n\n"

• hie-bios branch with explicit hie.yaml, without stack-yaml and with the same test suite name in both subpackages:

cradle:
  multi:
    - path: ./package1/
      config:
        cradle:
          cabal:
            component: "lib:package1"
    - path: ./package1/
      config:
        cradle:
          cabal:
            component: "package1:example-test" # test:example-test doesn't work
    - path: ./package1/
      config:
        cradle:
          cabal:
            component: "exe:package1"
    - path: ./package2/
      config:
        cradle:
          cabal:
            component: "lib:package2"
    - path: ./package2/
      config:
        cradle:
          cabal:
            component: "package2:example-test" # test:example-test doesn't work
    - path: ./package2/
      config:
        cradle:
          cabal:
            component: "exe:package1"

• there is a notificacion in vscode: "lintCmd: no access to the persisted file."
• It works as expected in all source files of all components
• Hover responds

• the cabal-helper-helper branch trigger a build with stack or cabal (that can be lengthy), if that build fails the cradle initialization doesn't finish and hover doesnt responds (only waiting)
  • the question is if the builds are triggered from hie directly or from cabal-helper

[jneira]

After the previous tests i've tested again dhall-haskell

• Renaming all components with duplicated names to avoid errors
• With the cabal-helper-helper branch
• Ensuring the cradle init finishes correctly
• and.... hover doesn't respond :worried: , so there is another factor involved when handling dhall-haskell
• logs: https://gist.github.com/jneira/4a1007dd9a34f9dfd7b54aea6c1277d6

[jneira]

Ok, ive just tested again with the latest hie-bios and the behaviour is the same: the cradle seems to finish but hover hangs showing Loading :worried: Maybe it has too much modules (38)?

[mpickering]

No, 38 is certainly not too many. It works fine with GHC for example. It the previous log you posted still accurate?

[fendor]

If that was too many modules, then the branch would be unacceptable! On Linux, hie is responsive on the project

[jneira]

I've tested to load the subproject dhall-haskell/dhall, forcing hie to use it as project root dir creating a cabal.project inside and the behaviour is still the same :thinking:

[jneira]

I've tested dhall-haskell/dhall-yaml and it works correctly...

[jneira]

I've tested opening the project with persist-virtual-fix and fixing the keys of VFS, tracing the lookups over the VFS and the execution get stuck here:

2019-12-01 00:51:34.2444094 [ThreadId 4] - <--2--{"jsonrpc":"2.0","params":{"value":{"kind":"report","percentage":1,"message":"Dhall.Main"},"token":0},"method":"$/progress"}
2019-12-01 00:51:34.2773866 [ThreadId 47] - Cradle set succesfully
2019-12-01 00:51:34.2793857 [ThreadId 47] - persistVirtualFile': uri: Uri {getUri = "file:///D:/dev/ws/dhall/dhall-haskell/dhall/src/Dhall.hs"}

Being the traces:

persistVirtualFile' :: Core.LspFuncs Config -> Uri -> IO (Maybe FilePath)
persistVirtualFile' lf uri = do
  infoM "hie" $ "persistVirtualFile': uri: " ++ show uri
  fp <- Core.persistVirtualFileFunc lf (toNormalizedUri uri)
  infoM "hie" $ "persistVirtualFile': fp: " ++ show fp
  return fp

So the execution seems to hang in Core.persistVirtualFileFunc. 🤔

[jneira]

I've added more traces to haskell-lsp vfs and the execution reach the write of the file:

2019-12-01 16:26:17.0113298 [ThreadId 49] - persistVirtualFile': uri: Uri {getUri = "file:///D:/dev/ws/dhall/dhall-haskell/dhall/src/Dhall.hs"}
2019-12-01 16:26:17.0113298 [ThreadId 49] - persistVirtualFile:NormalizedUri "file:///D:/dev/ws/dhall/dhall-haskell/dhall/src/Dhall.hs"
2019-12-01 16:26:17.0123296 [ThreadId 49] - persistFileVFS: uri = NormalizedUri "file:///D:/dev/ws/dhall/dhall-haskell/dhall/src/Dhall.hs", virtual file name = "C:\\TEMP\\haskell-lsp13000\\-1466182975-0-Dhall.hs.hs"
2019-12-01 16:26:17.0123296 [ThreadId 49] - persistFileVFS: temp file does not exist, writing file

The function traced is persistFileVFS:

-- | Write a virtual file to a temporary file if it exists in the VFS.
persistFileVFS :: VFS -> J.NormalizedUri -> Maybe (FilePath, IO ())
persistFileVFS vfs uri =
  case Map.lookup uri (vfsMap vfs) of
    Nothing -> Nothing
    Just vf ->
      let tfn = virtualFileName (vfsTempDir vfs) uri vf
          action = do
            logm $ B.pack $ "persistFileVFS: uri = " ++ show uri ++ ", virtual file name = " ++ show tfn
            exists <- doesFileExist tfn
            unless exists $ do 
              logm $ "persistFileVFS: temp file does not exist, writing file"
              writeFile tfn (Rope.toString (_text vf))
      in Just (tfn, action)

But i dont see persistVirtualFile': fp: path\to\temp\file so i assume the execution hangs in Dhall.Lsp.Core.persistVirtualFile, where afaiu the write of the file is just at the end 🤔

[fendor]

@jneira does this still happen?

[jneira]

I am afraid it still does. I tested it just after merging hie-bios and with my actual version which is not last master. I'll test with it just in case.

[jneira]

I've tested it with master (072ff7f) and i least i've got an error message (not sure if it was logged before):

2019-12-29 20:50:55.9845886 [ThreadId 52] - Cradle set succesfully
2019-12-29 20:50:55.9875878 [ThreadId 52] - haskell-lsp:persistFileVFS: Writing virtual file: uri = NormalizedUri "file:///D:/dev/ws/dhall/dhall-haskell/dhall/src/Dhall/Core.hs", virtual file = "C:\\TEMP\\haskell-lsp1056\\Core.hs-00000--2147427451.hs"
2019-12-29 20:50:56.0075737 [ThreadId 26] - Scheduler thread exited unexpectedly: C:\TEMP\haskell-lsp1056\Core.hs-00000--2147427451.hs: commitBuffer: invalid argument (invalid character)

Inspecting the vfs file we can observe that it is truncated just in this line: https://github.com/dhall-lang/dhall-haskell/blob/96e694d7c6368a327a24dc31e8dd870ae76e7272/dhall/src/Dhall/Core.hs#L133 That line has the first occurence of a utf-8 non ascii char ⊦, so it seems to be the main suspect

[fendor]

wow... that is awesome, right? You found a major clue. Is that the same issue repeatedly?

[jneira]

Yeah, i hope there will not be another hidden problem. I can reproduce it in a simple project simply adding a non-ascii char in the source file.

[fendor]

wait, in the source file? Ugh. Must be a haskell-lsp issue right? cc @alanz, @bubba

[jneira]

Yeah, the issue is triggered when the haskell source file has an unicode char in it. The client is sending the file correctly i think:

2019-12-29 21:16:07.7434497 [ThreadId 3] - ---> {"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///d%3A/dev/ws/haskell/cabal-test/src/Lib.hs","languageId":"haskell","version":1,"text":"module Lib where\r\nimport qualified Data.ByteString.Char8 as BS\r\n{--\r\n> ⊦ Type : Kind\r\n--}\r\ntest :: BS.ByteString\r\ntest = BS.pack \"Hello Haskell\"\r\n\r\ntest2 :: [a]\r\ntest2 = ([])"}}}