cabal: streamingProcess: runInteractiveProcess: exec: does not exist (No such file or directory)

[teto]

hi,

I am trying to update a professionnal codebase to use ghcide 0.2 instead of 0.1. We use bazel to compile our code. I could bump ghcide v 0.2 but when I run it at the root of the project I get errors about a missing cabal (I think, it is indeed absent). Because we have our own hie.yaml, I thought it could work without cabal ?

Errors:

ghcide
INFO: Invocation ID: 75d4f840-83da-4801-99c2-06ae3d36207d
INFO: Build option --compilation_mode has changed, discarding analysis cache.
INFO: Analyzed target @ghcide//:ghcide (5 packages loaded, 4022 targets configured).
INFO: Found 1 target...
Target @ghcide//:ghcide up-to-date:
  bazel-bin/external/ghcide/_install/bin/ghcide
INFO: Elapsed time: 1.227s, Critical Path: 0.12s
INFO: 0 processes.
INFO: Build completed successfully, 4 total actions
ghcide version: 0.2.0 (GHC: 8.8.3) (PATH: /home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/execroot/toplevel_workspace/bazel-out/k8-opt/bin/external/ghcide/_install/bin/ghc
ide)
Ghcide setup tester in /home/teto/nova/<PROJECT>.
Report bugs at https://github.com/digital-asset/ghcide/issues

Step 1/4: Finding files to test in /home/teto/nova/<PROJECT>
Found 2805 files

Step 2/4: Looking for hie.yaml files that control setup
Found 15 cradles

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
....

[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Making new HscEnv[main]
[INFO] Consulting the cradle for "/home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/external/stackage_ghcide/syb-0.7.1/src/Generics/SYB/Builders.hs"
> Loading: 0 packages loaded
> Loading: 35 packages loaded
> Loading: 35 packages loaded
[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Making new HscEnv[main]
[INFO] Consulting the cradle for "/home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/external/stackage_ghcide/ghcide-0.2.0/src/Development/IDE/Plugin/CodeAction/RuleTypes.hs"
File:    
  /home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/external/stackage_ghcide/ghcide-0.2.0/src/Development/IDE/Plugin/CodeAction/RuleTypes.hs
Hidden:   no
Range:    1:0-100001:0
Source:   compiler
Severity: DsError
Message: 
  cabal: streamingProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
File:    
  /home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/external/stackage_ghcide/monoid-subclasses-1.0.1/src/Data/Semigroup/Cancellative.hs
Hidden:   no
Range:    1:0-100001:0
Source:   compiler
Severity: DsError
Message: 
  cabal: streamingProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
File:    
  /home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/external/rules_haskell/tests/version-macros/HsLib.hs
Hidden:   no
Range:    1:0-100001:0
Source:   compiler
Severity: DsError
Message: 
  cabal: streamingProcess: runInteractiveProcess: exec: does not exist (No such file or directory)

and our hie.yaml

# Configuration for hie/ghcide.
# See https://github.com/mpickering/hie-bios/blob/master/README.md for the
# documentation
cradle:
  bios:
    program: ".hie-bios"
    dependency-program: ".hie-bios-dependencies"

with .hie-bios returning a bunch of ghc flags. First I wonder about the Found 15 cradles in the log since I define only one in hie.yaml ? It would be nice if Looking for hie.yaml files that control setup could give which one it selected or allow to debug the cradles. Is there some flag to dump more information ?

Thanks for the awesome work. Haskell-ide-engine and ghcide are absolutely fantastic for the haskell ecosystem.

[ndmitchell]

Agreed, we should say which hie.yaml file is being used.

Are there nested hie.yaml files in your repo? For example, I see ghcide fails, and ghcide has its own nested hie.yaml file at the root of its repo which says to use Cabal, which would remain if you import Stackage as its the case that its in our extra-source-files.

[pepeiborra]

Bios cradles have changed in hie-bios 0.5.0, in case you have missed it:

1. Instead of sending the ghc flags to stdout, they are now expected to write them to a file path controlled by the HIE_BIOS_OUTPUT env var
2. In addition to the list of flags, they must also enumerate all the modules in the project

See https://github.com/mpickering/hie-bios/tree/0.5.0 for more details

[teto]

thank you very much for these answers, it already unblocked me a bit: I've gone from 3 to 50 files scanned (out of ~3000) by installing cabal and writing to the file designed by $HIE_BIOS_OUTPUT.

I am sorry for the long post/to take up some of your time, I usually worked on cabal projects (which worked out of the box or pretty close) and this is my first time dabbling in hie-bios etc.

There is only one hie.yaml in the folder so I wonder where the "15" cradles come from ?

I noticed a difference between 0.1 and 0.2 start, ghcide 0.1 would display:

Step 3/6, Cradle 1/12: Implicit cradle for /home/teto/nova/jinko
Cradle {cradleRootDir = "/home/teto/nova/jinko", cradleOptsProg = CradleAction: Bios}

with 0.2, it's just

Step 2/4: Looking for hie.yaml files that control setup
Found 15 cradles

so I wonder if 0.2 skipped the hie.yaml for some reason ? the .hie-bios program does seem to be called though so maybe it's just a change in the output of ghcide. Also it seems to be run for each file which was not the case before.

@pepeiborra just to make sure, is such a .hie-bios-output valid (there is no example on hie-bios wiki)?

-no-user-package-db
-hide-all-packages
-package-id
aeson-1.4.7.1
-package-id
attoparsec-0.13.2.4
-package-db
bazel-out/k8-fastbuild/bin/external/hackage/lrucaching-0.3.3/_install/lrucaching-0.3.3.conf.d
-package-db
bazel-out/k8-fastbuild/bin/simwork/utils/simworkZSutilsZSutils
-lblas
-llapack
-llapacke
-lz
-Lbazel-out/k8-fastbuild/bin/_solib_k8/_U@liblapack_S_S_Clib___Uexternal_Sliblapack_Slib+
-Lbazel-out/k8-fastbuild/bin/_solib_k8/_U@zlib.dev_S_S_Czlib___Uexternal_Snixpkgs_Uzlib_Slib
-Lbazel-out/k8-fastbuild/bin/_solib_k8/_U@zlib.dev_S_S_Czlib___Uexternal_Snixpkgs_Uzlib_Slib
-pgma
bazel-out/k8-fastbuild/bin/external/rules_haskell/haskell/cc_wrapper-python
-pgmc
bazel-out/k8-fastbuild/bin/external/rules_haskell/haskell/cc_wrapper-python
-pgml
bazel-out/k8-fastbuild/bin/external/rules_haskell/haskell/cc_wrapper-python
-pgmP
bazel-out/k8-fastbuild/bin/external/rules_haskell/haskell/cc_wrapper-python -E -undef -traditional
-optc-fno-stack-protector
-isimwork/scoring/src
-ibazel-out/k8-fastbuild/bin/simwork/scoring/src
-freduction-depth=0
-fwrite-ide-info
-Wall
-Werror
-Wcompat
-Wincomplete-record-updates
-Wincomplete-uni-patterns
-Wredundant-constraints
-Wnoncanonical-monad-instances
-XApplicativeDo
-XBangPatterns
-XConstraintKinds

I am a bit alarmed by the number of main in HscEnv (not sure if that's normal) while the latter error hie-bios.sh may be due to our packaging. It seems strange to me that it would search for it in the same folder as the hie.yaml aka /home/teto/nova/jinko2.

[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Using interface files cache dir: /home/teto/.cache/ghcide/main-6e77eacc516754c626b5a5b7f91bbe7920e821e8
[INFO] Making new HscEnv[main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,ma
in,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main
,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main,main]
[INFO] Consulting the cradle for "/home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/external/stackage_ghcide/hie-bios-0.5.0/tests/projects/simple-bios/B.hs"
File:    
  /home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/external/ghcide/src/Development/IDE/Core/OfInterest.hs
Hidden:   no
Range:    1:0-100001:0
Source:   compiler
Severity: DsError
Message: 
  /home/teto/nova/jinko2/hie-bios.sh: streamingProcess: runInteractiveProcess: exec: does not exist
  (No such file or directory)
File:    
  /home/teto/.cache/bazel/_bazel_teto/f3097620197e6c203645e1230d1ae88c/external/stackage_ghcide/hie-bios-0.5.0/tests/projects/simple-bios/B.hs

[pepeiborra]

As I mentioned, your bios script also needs to list all the modules in your project, immediately after the list of flags

[teto]

sry: so basically it should look like

...
-XApplicativeDo
-XBangPatterns
-XConstraintKinds
./myModule/main.hs
./myModule/Module1/Resource.hs
...

? can one use wildcards ?

http://hackage.haskell.org/package/hie-bios-0.5.1 doesn't mention the list of modules (or is it included in "list of options" ?) he bios program should consult the HIE_BIOS_OUTPUT env var and write a list of options to this file separated by newlines. Once the program finishes running hie-bios reads this file and uses the arguments to set up the GHC session. This is how GHC's build system is able to support hie-bios.

Thanks !

[pepeiborra]

I am pretty sure that the list of modules is required and wildcards are not supported, but my knowledge ends there. Certainly looks like the hie-bios README needs to be updated. /cc @mpickering @bubba @fendor

[fendor]

Building without cabal is definitely supported.

Would you mind executing hie-bios debug <path to a sourcefile> in your project root and link the logs? The hie-bios executable can be installed from https://github.com/mpickering/hie-bios.

Judging from just the report, i cant see a problem. I guess you can not share the project? Can you maybe share your rough project setup, etc...?

EDIT: I have just seen the arguments. Indeeed, the targets are missing.

[teto]

I was not sure how to have bazel export absolute paths (instead of the //myProject:libraryPathToModule) so I used this snippet to append

    hie_bios_modules() {
      bazel query 'kind("source file", deps("//project"))' 2>&1|grep myProject| tr : / | sed -e 's|//|/|'
    }

   hie_bios_flags  > $HIE_BIOS_OUTPUT  
   hie_bios_modules >> $HIE_BIOS_OUTPUT  

It seems to work so far but I do miss seeing debug info about the cradle configurations from ghcide.

Now I am looking into using multicradle (very useful for our project with ten libraries) but I guess I will post to hie-bios instead.

Thanks to everyone for the reactivity/help.