Index src file in mods using information from hie file

[josephsumabat]

Currently hiedb does not index src files in the mods directory making it difficult to get the hie file given a src file without resorting to file manipulation since a fake file is passed right now (lookupHieFileFromSource does not work). This PR fetches the src file from the hiefile which is passed to the addRefsFromLoaded and should always exist.

Before:

After:

[wz1000]

This doesn't seem right to me because the path in hie_hs_file is relative to the path of working directory of the GHC process that produces the .hie file, and may not exist anymore (for example cabal compiles in a temporary directory which is subsequently removed). This distinction between "this .hie file is associated with a source file that is guaranteed to be available because it is part of the users project" vs "this .hie file was at one point associated with a source file but we don't know if that file still exists because we don't manage it" is what RealFile and FakeFile tries to capture.

There is an additional complication with FakeFile where it can optionally have a FilePath which points to a "read only" copy of a source that might be written out by a caller (for example HLS) to allow users to "go to definition" to a symbol outside their project. In this case, the file is treated as "fake" because we can't recompile the dependency that this file is associated with.

[josephsumabat]

This doesn't seem right to me because the path in hie_hs_file is relative to the path of working directory of the GHC process that produces the .hie file, and may not exist anymore (for example cabal compiles in a temporary directory which is subsequently removed). This distinction between "this .hie file is associated with a source file that is guaranteed to be available because it is part of the users project" vs "this .hie file was at one point associated with a source file but we don't know if that file still exists because we don't manage it" is what RealFile and FakeFile tries to capture.

There is an additional complication with FakeFile where it can optionally have a FilePath which points to a "read only" copy of a source that might be written out by a caller (for example HLS) to allow users to "go to definition" to a symbol outside their project. In this case, the file is treated as "fake" because we can't recompile the dependency that this file is associated with.

I see, thanks so much for the explanation! I apologize for my naiivety around the case. Do you think using the hie_hs_file is an unworkable solution or would it be possible to use it with a check for if the directory exists? Something along the lines of: c23c3cb2a8adc1f124646e1e7d68e7fad9423c67

There probably isn't a good way to detect the read only fake file case using this information though so I understand if this method can't be made to work as well.

[wz1000]

What is the use case you are wanting to support?

[josephsumabat]

Ah for context I work at a company with a fairly large code-base which unfortunately HLS tends to run out of memory on

As a workaround I have a small project that I've been working on that uses the information from hiedb as a language server (https://github.com/josephsumabat/static-ls/tree/init). My ideal UX would be to be able to use hiedb or ghcid on top of hiedb to index the source files.

When given a source file by TextDocumentIdentifier I retrieve the hie file information by manipulating the path which works for now but is unfortunately not a very robust solution or indexing the files on startup which is very slow and source file information by individually reading hie files which is slow for large calls to findReferences

The main functionality I'm looking for is being able to query using lookupHieFileFromSource from a hiedb -D .hiedb index .hiefiles/

[wz1000]

I would say the way to proceed would be to add an option to the hiedb index command that accepts a base directory, for instance hiedb index --src-base-dir /path/to/base/cir which will index the files as RealFiles, treating them as relative to the base directory supplied if they are relative paths.

[josephsumabat]

Thank you for the advice, I will write up a PR for that instead!