Commands timeout while program is running in the REPL

[greghendershott]

From issue #126, @samertm reports:

Also, if something is running in *Racket REPL*, racket-mode will just constantly stall. So if you call racket-run in a buffer with this:

#lang racket
(require web-server/servlet web-server/servlet-env)
(define (start req) (response/xexpr "test"))

(serve/servlet start)

Without closing the server, if you start writing "(define " (or anything else), racket-mode will stall. Not sure if these two bugs are related. What function is being called when you type "(some-chars " (with the space)?

(This is with eldoc-mode enabled.)

My explanation:

Basically, the communication mechanism is that the edit buffer "types things for you" into the REPL buffer. At the REPL prompt, if you type ,help you will many of these "commands". For example you could type ,sig foo and see the function signature (if available) for foo. When the eldoc code needs a sig, basically that is what's happening behind the scenes.

As a result, unless the REPL is "ready" at a prompt, the commands can't be sent. (You can C-c C-c in the REPL to break the running Racket program. I mean that the edit buffer can't send the "hidden commands".)

Generally this works well and is very reliable (there are no thread synchronization issues for example). Except in the use case you described: Keeping something like a server running a long time in the REPL while continuing to edit.

Probably I should redesign this to use a socket and work out how to synchronize things and efficiently handle programs that emit a huge amount to stdout, etc. etc.

[greghendershott]

Working on this in the command-socket branch.

Currently this is simply a TCP connection "side channel" for Elisp to send commands and receive responses from the Racket code. The Racket REPL is still a conventional comint-mode buffer. (In other words this is not attempting to be some completely redesigned "network REPL", where even stdin and stdout go over TCP.)

In my limited testing this is working well, so far. It does handle the use-case above, where the REPL is not at a command prompt because it is still evaluating something.

[However it does have the limitation that, because we're still inside the namespace-require of the module, we haven't yet reached the point where we can (current-namespace (module->namespace x)). In other words, commands like syms, def, and describe will only know about symbols in racket/base. Ditto for racket-mode commands depending on those, like completion, C-c C-., and M-. -- they won't work for things defined in that file. I don't see any good way around that, short of rewriting/annotating the module.]

Rebasing the debugger branch to this will be interesting. Presumably it will need 1 (maybe 2) additional TCP connections, to handle the extra things it is currently using files for.

[tbrooke]

Just a thumbs up on this running into the same problem using web-server - the work around seem to be to kill the repl before returning to editing

[greghendershott]

Unfortunately I won't be able to change this in the foreseeable future.

One work-around I can suggest is to use a thread.

For example replace (serve/servlet start) with (thread (λ _ (serve/servlet start))) or equivalent (thread (thunk (serve/servlet start))).

Although the web server banner output may appear after the prompt...

foo.rkt> Your Web application is running at http://localhost:8000/servlets/standalone.rkt.
Stop this program at any time to terminate the Web Server.

...you do have control. You can e.g. type 42 and press ENTER. It will evaluate and print 42, and give you another prompt.

[greghendershott]

By the way: The next racket-run will delete this extra thread. racket-mode uses a custodian for all the resources created for/by the REPL. All are released/freed.