Update editor integration to new loading approach

[chrisdone]

Opening this issue to notify interested users and authors of editor bindings to stack-ide. This update requires you to update both stack and stack-mode. stack-ide the binary itself has no changes, but you can update anyway if you like.

The way loading is done with stack-ide has changed now. Initially, it would work like this:

1. stack-ide is launched by stack, and stack sends the list of load targets (modules) to load.
2. stack-ide would then load these modules and report errors.
3. Your editor would send RequestUpdateSession with empty contents.

Now, it's like this:

1. stack ide is launched by the editor via stack ide start <package-target> (that can be acquired by stack ide packages).
2. Your editor should call stack ide load-targets <package-target> to get a list of modules that are in the project.
3. Your editor then sends RequestUpdateSession when it wants to load module(s):

{
    "tag": "RequestUpdateSession",
    "contents": [
        {
            "tag": "RequestUpdateTargets",
            "contents": {
                "tag": "TargetsExclude",
                "contents": [
                    "src/Control/Concurrent/Execute.hs",
                    "src/Stack/Package.hs"
                ]
            }
        },
        {
            "tag": "RequestUpdateTargets",
            "contents": {
                "tag": "TargetsInclude",
                "contents": [
                    "src/Stack/Package.hs"
                ]
            }
        }
    ]
}

This request will load everything necessary for Stack.Package and no more. The two target updates are monoidal, so the latter overrides the former (this is why you can specify Package.hs in both, a small convenience). This is much faster than the previous approach of loading everything every time. When you do want to load everything, you can put the complete list of modules in TargetsInclude and omit TargetsExclude entirely.

Reference implementation in Emacs here: https://github.com/commercialhaskell/stack-ide/blob/master/stack-mode/stack-mode.el#L892-L911

In stack-mode this keybinding is now:

• C-c C-l - load the current module.
• C-c C-c - compile all modules.

This can be seen as analogous to :l in GHCi vs cabal build. The flycheck behaviour uses the C-c C-l case -- it only compiles the current module. When you want to check all modules, C-c C-c will trigger flycheck for all modules.

Pinging @lukexi (stack-ide-sublime author).

Pinging @snoyberg @mgsloan @LukeHoersten @dysinger @CRogers.

[lukehoersten]

Don't forget to update the docs and commentary with the additional keybindings =)

[lukehoersten]

I'm getting a regression:

<unknown>:(0,0)-(0,0): 
can't find file: /Users/luke/Code/haskell/project/Invalid argument `load-targets'

[chrisdone]

Did you update stack?

[lukehoersten]

Yes

[lukehoersten]

Another regression that may be related to the first: My package has a library and executable that depends on the library. The executable code is having errors reported that it cant find any of the library modules.

[lukehoersten]

I've re-installed stack and stack-ide numerous times now just to be sure. There's definitely a regression where modules in executable sections can't see modules from the library section that are build-depped and imported properly.

[lukehoersten]

I've been updating stack from stack install stack - should I be running from git?

[wraithm]

I tried using stack from git (stack upgrade --git), and emacs just hangs when I try to use that version of stack whenever I try to load code.

However, with these versions, stack ide starts properly and seems to load the code.

$ stack-ide --version
Version 0.1.0.0, Git revision 7e93bd3e03502beafb4613b7bc690fb9f0db1314 (163 commits) X86_64
$ stack --version 
Version 0.1.4.0, Git revision 165db5bd77cc88745e0ec8e143275575e5a75d93 (1723 commits) X86_64

I get exactly the same thing as @LukeHoersten when I do C-c C-c in a new buffer:

<unknown>:(0,0)-(0,0):
can't find file: /home/matt/project/Invalid argument `load-targets'

[wraithm]

Okay, I'm trying to debug the git version of stack again. This version has the stack ide load-targets command. The ide process clearly loads, and I see stuff in the stack:project buffer. However, I don't see the project doing anything when I do C-c C-c or C-c C-l. Nothing pops up in the stack:project buffer. Emacs hangs on C-c C-t or C-c C-i.

[lukehoersten]

It might be that it's doing lots of slow flycheck compiling or something. I see this in my *stack:...* buffer:

Running Flycheck with Stack backend ...
 <- Progress: 0/2             statistics-0.13.2.3: configure
 <- Progress: 0/2             statistics-0.13.2.3: build
 <- Progress: 0/2             statistics-0.13.2.3: install
 <- Progress: 0/2             Progress: 1/2             criterion-1.1.0.0: configure
 <- Progress: 1/2             criterion-1.1.0.0: build
 <- Progress: 1/2             criterion-1.1.0.0: install
 <- Progress: 1/2             Completed all 2 actions.

There are a whole bunch of ^H characters in the whitespace there. This is something I haven't seen before.

[rvion]

I'm trying to update subl-stack-ide to that new method, but I get "Stack-IDE error: stty: stdin isn't a terminal"

related code is near https://github.com/lukexi/stack-ide-sublime/blob/master/stack-ide.py#L610

could it be some bug ?

[chrisdone]

@LukeHoersten

I've been updating stack from stack install stack - should I be running from git?

Yes, this is just on master.

[chrisdone]

@WraithM

can't find file: /home/matt/project/Invalid argument `load-targets'

This means you're not using a recent stack. It should be like this:

bash-3.2$ stack --version
Version 0.1.4.1, Git revision 63b4eca12e99e74d22e5df73f7e9888585f4cea3 (dirty) (1878 commits) X86_64
bash-3.2$ stack ide load-targets stack
src/Control/Concurrent/Execute.hs
src/Data/Aeson/Extended.hs
src/Data/Attoparsec/Args.hs
...

[chrisdone]

Specifically you should have this commit which is on master.

[chrisdone]

@rvion What OS are you on?

[chrisdone]

@rvion Retry master now, I pushed 51e73e6 which should ignore failure in the case it's not a terminal.

[rvion]

@chrisdone I'm on Mac I confirm that stack ide works weird even on the command line

when I start stack ide from terminal

$ stack ide start ride
{"tag":"ResponseWelcome","contents":[0,1,1]}
{"tag":"ResponseLog","contents":"[Info] Initializing ide-backend session @(ideba_HqZKAqZV4wa5sqhSD7XjZv:IdeSession.Update ./IdeSession/Update.hs:195:3)\n"}
{"tag":"ResponseLog","contents":"[Debug] Session dir = /Users/RemiVion/dev/ride/session.98045 @(ideba_HqZKAqZV4wa5sqhSD7XjZv:IdeSession.Update ./IdeSession/Update.hs:208:3)\n"}
{"tag":"ResponseLog","contents":"[Debug] Dist dir = /Users/RemiVion/dev/ride/.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/ @(ideba_HqZKAqZV4wa5sqhSD7XjZv:IdeSession.Update ./IdeSession/Update.hs:210:3)\n"}
{"tag":"ResponseLog","contents":"[Debug] Local working dir = /Users/RemiVion/dev/ride/ @(ideba_HqZKAqZV4wa5sqhSD7XjZv:IdeSession.Update ./IdeSession/Update.hs:217:17)\n"}
{"tag":"ResponseLog","contents":"[Info] Searching for ide-backend-server on this path: /Users/RemiVion/dev/ride/.stack-work/install/x86_64-osx/lts-3.4/7.10.2/bin/:/Users/RemiVion/.stack/snapshots/x86_64-osx/lts-3.4/7.10.2/bin/:/Users/RemiVion/dev/neo4j-community-2.2.5/bin:/Users/RemiVion/.local/bin:/Users/RemiVion/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin @(ideba_HqZKAqZV4wa5sqhSD7XjZv:IdeSession.RPC.Client ./IdeSession/RPC/Client.hs:341:5)\n"}
{"tag":"ResponseLog","contents":"[Info] Found ide-backend-server - using this one: /Users/RemiVion/.local/bin/ide-backend-server @(ideba_HqZKAqZV4wa5sqhSD7XjZv:IdeSession.RPC.Client ./IdeSession/RPC/Client.hs:343:5)\n"}
{"tag":"ResponseLog","contents":"[Debug] Recompile required, starting... @(ideba_HqZKAqZV4wa5sqhSD7XjZv:IdeSession.Update.ExecuteSessionUpdate ./IdeSession/Update/ExecuteSessionUpdate.hs:223:5)\n"}

Then, at the very moment I finish typing {"tag": "RequestUpdateSession", "contents": []}, right after I typed the last }, without pressing enter, I get the rensponse from stack:

{"tag": "RequestUpdateSession", "contents": []}{"tag":"ResponseLog","contents":"[Debug] Recompile required, starting... @(ideba_HqZKAqZV4wa5sqhSD7XjZv:IdeSession.Update.ExecuteSessionUpdate ./IdeSession/Update/ExecuteSessionUpdate.hs:223:5)\n"}
{"tag":"ResponseUpdateSession","contents":{"tag":"UpdateStatusDone","contents":[]}}

---

when you type some specific chars (even when not pressing enter) stack-ide crash eg:

[[A{"tag":"ResponseFatalError","contents":"user error (Failed reading: satisfy)"}
Process exited with ExitFailure 1: /Users/RemiVion/.local/bin/stack-ide --verbose --include=/Users/RemiVion/dev/ride/:/Users/RemiVion/dev/ride/src/:/Users/RemiVion/dev/ride/.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/autogen/:/Users/RemiVion/dev/ride/.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/ --local-work-dir=/Users/RemiVion/dev/ride/ --ide-backend-tools-path=/Users/RemiVion/dev/ride/.stack-work/install/x86_64-osx/lts-3.4/7.10.2/bin/:/Users/RemiVion/.stack/snapshots/x86_64-osx/lts-3.4/7.10.2/bin/:/Users/RemiVion/dev/neo4j-community-2.2.5/bin:/Users/RemiVion/.local/bin:/Users/RemiVion/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin --dist-dir=/Users/RemiVion/dev/ride/.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/ --ghc-option=-hide-all-packages --ghc-option=-i/Users/RemiVion/dev/ride/ --ghc-option=-i/Users/RemiVion/dev/ride/src/ --ghc-option=-i/Users/RemiVion/dev/ride/.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/autogen/ --ghc-option=-i/Users/RemiVion/dev/ride/.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/ --ghc-option=-stubdir=/Users/RemiVion/dev/ride/.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/ --ghc-option=-optP-include --ghc-option=-optP/Users/RemiVion/dev/ride/.stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/autogen/cabal_macros.h --ghc-option=-package=base --ghc-option=-package=transformers-0.4.2.0 --ghc-option=-package=lens-4.12.3 --ghc-option=-package=containers-0.5.6.2 --ghc-option=-package=text-1.2.1.3 --ghc-option=-package=Spock-0.8.1.0 --ghc-option=-package=wai-middleware-static-0.7.0.1 --ghc-option=-package=wai-extra-3.0.10 --ghc-option=-package=aeson-0.8.0.2 --ghc-option=-package=process-1.2.3.0 --ghc-option=-package=filemanip-0.3.6.3 --package-db=/Users/RemiVion/.stack/snapshots/x86_64-osx/lts-3.4/7.10.2/pkgdb/:/Users/RemiVion/dev/ride/.stack-work/install/x86_64-osx/lts-3.4/7.10.2/pkgdb/

it also seems to happen when stdin is not a tty under some other circumstances

[lukehoersten]

@chrisdone @WraithM running against the latest git versions of stack and stack-ide causes stack-mode/emacs to lock up on stack-mode-load etc.

[chrisdone]

The stack-mode-load function should not exist anymore on the latest stack-ide stack-mode. There's stack-mode-load-module and stack-mode-load-all-modules now. If you're not using them you need to update or restart Emacs or something.

[chrisdone]

@rvion that's okay, as long as it works as the piped process it shouldn't matter.

[lukehoersten]

@chrisdone I don't know it's calling stack-mode-load specifically, I just know it locks up as soon as stack-mode is enabled on the buffer. stack-mode-load, or whatever the equivalent is now, I'm assuming is where it's locking up emacs.

This is happening with stack and stack-ide from latest git.

[lukehoersten]

@chrisdone moving to it's own issue here with latest information: https://github.com/commercialhaskell/stack-ide/issues/87

[rvion]

all's working for me now