Extended recommendations for large projects

[Benjamin-McRae-Tracsis]

As Mercury seem to have, I also have a large monorepo of a haskell project, which HLS cannot properly handle.

I'm currently struggling to get static-ls to work for my system.

I've attached the setup for static-ls to my pedantic build command, so I can explore it before offering it to the rest of my team; that looks as follows:

stack test --no-run-tests --work-dir .stack-work-pedantic --fast --haddock --ghc-options="-j4 -ddump-minimal-imports -Werror -fwrite-ide-info -hiedir .hiefiles -hidir .hifiles -fdefer-type-errors -Werror=deferred-type-errors -Werror=deferred-out-of-scope-variables -fno-defer-typed-holes -package-db /home/myname/.cabal/store/ghc-9.8.2-6af5/package.db -plugin-package hiedb-plugin -fplugin Plugin.HieDb" --bench --no-run-benchmarks "$@"

However, static-ls does not seem to be working. From what I can tell, the above command makes the .hiefiles folders go into the sub directories for each package, meaning that they cannot be access from the top level static-ls running.

I tried to use --hie-files, but I'm struggling to make this work with alloglot.

Would I be able to have some additional guidance on what to do?

Setup notes:

• hiedb-plugin was installed using cabal install --lib globally to avoid adding the depedency to the project for now
• compilation mostly succeeds except for one troublesome package which has often given issue in the past for its linking
• static-ls seems to run with alloglot but doesn't do anything. when I amend the serverCommand to static-ls --hie-files ./a-package/.hiefiles to test it out, it understandably fails because that's not a single executable for it to start up.
• I use GHC 9.8.2, and compiled static-ls for my system by cloning the project and bumping the deps, which seemed to compile fine.

[josephsumabat]

Ah sub-directories are a bit tricky on vs-code. You're correct that without additional configuration it's attempting to search the top level workspace file for hiefiles, src files, etc.

We will push an update that adds a yaml or json configuration to avoid the argument issue on alloglot likely within the next week or two. Let me see if I can get the multiple package case working on a toy project by wrapping the command in a simple script in the meantime

[Benjamin-McRae-Tracsis]

That's appreciated, keep me posted! I'll leave this open for now, but I can open a linked issue in alloglot if that would be more helpful.

Note that I'm not sure that being able to add arguments to alloglot would be sufficient, because I need to be able to add .hiefiles from each build directory, and I'm not sure if globs or multiple --hiefiles args are supported.

[josephsumabat]

Great point - I'll add support for multiple hie file directories too

[josephsumabat]

Oh there is one other work-around too for the time being -

find . -name .hiefiles | awk '{print $1" "substr($1,1,length($1)-length(".hiefiles"))}' | xargs -n2 sh -c 'hiedb -D .hiedb index $0 --src-base-dir=$1 --skip-types'

The following script can index all hiefiles in sub-directories into a single .hiedb - this can allow functionality on a multi-project root

[Benjamin-McRae-Tracsis]

I had some time to continue tinkering, and after re-refreshing with a big build (and removing -hidir .hifiles), and running your provided indexing command, I can see that static-ls mostly works with alloglot and vscode!

I'm missing documentation probably because the -hidir doesn't work for some reason, and sometimes when jumping to definition with ctrl+click my editor tries to open src/ParentModule/SubModule.hs instead of package-name/src/ParentModule/SubModule.hs. It's also very obviously less featureful, but it's nice that it continues to "work" while I'm making changes, which is a change from HLS going "goodbye, see you in a week".

[josephsumabat]

Hm I'll look into the hidir issue more - for the ctrl+click issue you may be able to modify the script a bit if you modify the first two commands:

find . -name .hiefiles | awk '{print $1" "substr($1,1,length($1)-length(".hiefiles"))}'

you want to save the .hiefiles in the same location as the src file if they are in different locations you would need to modify the command to print out the "base directory".

[Benjamin-McRae-Tracsis]

Maybe it's too early to tell, but deleting the top level .hiedb and rerunning everything seems to result in a more consistent "jump to" experience. I'll keep you posted.

[Benjamin-McRae-Tracsis]

Still found one occurrence of a bad jump-to, odd that it was mostly fixed after a complete rebuild.

[josephsumabat]

Was this that it was jumping to the wrong (likely stale) location or that the jump to was not working? There are some recompilation issues i know of with ghc's recompilation checker not recompiling if whitespace was added for example which could cause the destination file to be stale.

[Benjamin-McRae-Tracsis]

jumping to the wrong location, that of src/module/module.hs instead of package/src/module/module.hs

[Benjamin-McRae-Tracsis]

Tried to get hifiles working again, and read the error messages more closely.

a-package   > Error: Cabal-simple_HwdwpEmb_3.10.2.0_ghc-9.8.2: Could not find module:
a-package   > A.Module.Name with any suffix: ["hi"] in the search path:
a-package   > [".stack-work-pedantic/dist/x86_64-linux/ghc-9.8.2/build"]

looks like ghc/Cabal-library was struggling with the same thing that static-ls was, and it can't find the hifiles folders.

[josephsumabat]

Do you have any other flags (for example dynamic linking?). If you have the build directory you can directly symlink the hifiles instead and not use that flag:

#! /bin/sh

# Symlinks hi files from cabal to the .hifiles directory allowing for access from tooling

if [ ! -e .hifiles ]; then
  suffix="/x/mwb/noopt/build/mwb/mwb"
  # NOTE: This takes 10 seconds!
  cabal_exec_dir=$(_cabal list-bin mwb)
  # NOTE: This directory may not exist until you run a build.
  cabal_hi_dir=${cabal_exec_dir%"$suffix"}/noopt/build

  ln -sf "$cabal_hi_dir" "$PWD/.hifiles"
fi;

This script should work for cabal (maybe stack too since it uses cabal beneath the hood - but in theory you could do something similar if it doesn't)

[Benjamin-McRae-Tracsis]

I don't think we do much dynamic linking bar one package right near the end. the only flag that could be is -msse2.4, which I don't think is relevant.

I'll adjust the script you've given above to work with multiple packages and report back after.

[Benjamin-McRae-Tracsis]

I went with find . -wholename "*/.stack-work-pedantic/dist/x86_64-linux/ghc-9.8.2/build" | awk '{print $1" "substr($1,1,length($1)-length(".stack-work-pedantic/dist/x86_64-linux/ghc-9.8.2/build"))}' | xargs -n2 sh -c 'ln -sfL "$PROJ_BASE/$0/" "$1/.hifiles"', which seems to link up the hi files well enough. I fear that I'll need to do some kind of combining like you did for https://github.com/josephsumabat/static-ls/issues/127#issuecomment-2387428464, but I can't see a way to do that.

[josephsumabat]

Hm i think we would need something similar to the other issue like a list of directories we could search for - unfortunately we dont have a running index of hifiles for now.