Background process maybe holding state between REPL restarts

[vkz]

Hey Greg. This is a weird one which I'm not entirely sure how to report or reproduce reliably. Perhaps you'll be able to think of where the problem might be simply by knowing how sausage is made.

I had on several occasions the following happen:

• hack with Racket mode and its dedicated repl,
• run the code "externally" so it triggers compilation of the codebase and such,
• kill Emacs repl,
• racket-repl again and try to racket-run buffer that used to work,
• get bizarre errors like "module of some-dependency not found",
• blame it on compiled/ artifacts so remove all of those and restart repl
• still broken,
• kill Emacs and suddenly it works again.

This last step led me to check processes and turns out that killing REPL buffer doesn't actually kill the server process. That could be considered a feature and I guess I understand how "restarting" repl is so fast now, cause server keeps running. I would argue though, that this isn't what I expect to happen when I "kill my repl". First, I'd probably wish that was clearly documented and racket-repl-exit with universal prefix would make a prominent appearance in docs. Second, and that's a more alarming thing is that looks to me that the background process holds on to some state and that's what's been causing my issue. That's terrifying since I can no longer trust my REPL and defeats the whole "always start fresh" paradigm Racket is so proud of without my gaining any Clojure-like "liveness".

Unless you can think of where the problem might be, I would file it under "weirdness" until I can figure a way to reproduce reliably.

``` ((alist-get 'racket-mode package-alist)) ((emacs-version "26.3") (emacs-uptime "11 minutes, 16 seconds") (system-type darwin) (major-mode racket-mode) (racket--el-source-dir "/Users/russki/.emacs.d/lib/racket-mode/") (racket--rkt-source-dir "/Users/russki/.emacs.d/lib/racket-mode/racket/") (racket-program "racket") (racket-command-port 55555) (racket-command-timeout 10) (racket-memory-limit 2048) (racket-error-context medium) (racket-error-context medium) (racket-history-filter-regexp "\\`\\s *\\S ?\\S ?\\s *\\'") (racket-images-inline t) (racket-images-keep-last 100) (racket-use-repl-submit-predicate nil) (racket-images-system-viewer "open") (racket-pretty-print t) (racket-indent-curly-as-sequence t) (racket-indent-sequence-depth 0) (racket-pretty-lambda nil) (racket-smart-open-bracket-enable nil) (racket-module-forms "\\s(\\(?:module[*+]?\\|library\\)") (racket-logger-config ((cm-accomplice . warning) (GC . info) (module-prefetch . warning) (optimizer . info) (racket/contract . error) (sequence-specialization . info) (* . fatal)))) (enabled-minor-modes (auto-compile-on-load-mode) (auto-composition-mode) (auto-compression-mode) (auto-encryption-mode) (auto-fill-function) (auto-fill-mode) (auto-save-mode) (blink-cursor-mode) (column-number-mode) (diff-auto-refine-mode) (diff-hl-mode) (electric-indent-mode) (file-name-shadow-mode) (font-lock-mode) (global-auto-revert-mode) (global-diff-hl-mode) (global-eldoc-mode) (global-font-lock-mode) (global-launch-mode) (global-prettify-symbols-mode) (global-subword-mode) (global-undo-tree-mode) (helm-descbinds-mode) (helm-mode) (hes-mode) (hs-minor-mode) (keyfreq-autosave-mode) (keyfreq-mode) (launch-mode) (line-number-mode) (lispy-mode) (mouse-wheel-mode) (override-global-mode) (paredit-mode) (projectile-mode) (rainbow-delimiters-mode) (recentf-mode) (save-place-mode) (savehist-mode) (semantic-minor-modes-format) (server-mode) (sexy-mode) (shell-dirtrack-mode) (show-paren-mode) (smooth-scrolling-mode) (subword-mode) (temp-buffer-resize-mode) (transient-mark-mode) (undo-tree-mode) (winner-mode) (ws-butler-mode)) (disabled-minor-modes (abbrev-mode) (auto-compile-mode) (auto-compile-on-save-mode) (auto-image-file-mode) (auto-revert-mode) (auto-revert-tail-mode) (auto-save-visited-mode) (avy-linum-mode) (buffer-read-only) (cl-old-struct-compat-mode) (compilation-in-progress) (compilation-minor-mode) (compilation-shell-minor-mode) (completion-in-region-mode) (defining-kbd-macro) (delete-selection-mode) (diff-hl-dir-mode) (diff-minor-mode) (dired-hide-details-mode) (dired-isearch-filenames-mode) (dired-omit-mode) (diredp-breadcrumbs-in-header-line-mode) (display-time-mode) (edebug-mode) (eldoc-mode) (electric-layout-mode) (electric-pair-mode) (electric-quote-mode) (elisp-slime-nav-mode) (global-dired-hide-details-mode) (global-reveal-mode) (global-semantic-highlight-edits-mode) (global-semantic-highlight-func-mode) (global-semantic-show-parser-state-mode) (global-semantic-show-unmatched-syntax-mode) (global-semantic-stickyfunc-mode) (global-superword-mode) (global-visual-line-mode) (helm--minor-mode) (helm--remap-mouse-mode) (helm-adaptive-mode) (helm-autoresize-mode) (helm-migemo-mode) (helm-popup-tip-mode) (horizontal-scroll-bar-mode) (ibuffer-auto-mode) (ido-everywhere) (image-dired-minor-mode) (image-minor-mode) (isearch-mode) (ivy-mode) (jit-lock-debug-mode) (lispy-goto-mode) (lispy-other-mode) (menu-bar-mode) (next-error-follow-minor-mode) (org-cdlatex-mode) (org-make-toc-mode) (org-src-mode) (outline-minor-mode) (overwrite-mode) (paragraph-indent-minor-mode) (prettify-symbols-mode) (racket-debug-mode) (racket-smart-open-bracket-mode) (racket-xp-mode) (reveal-mode) (semantic-highlight-edits-mode) (semantic-highlight-func-mode) (semantic-mode) (semantic-show-parser-state-mode) (semantic-show-unmatched-syntax-mode) (semantic-stickyfunc-mode) (sh-electric-here-document-mode) (size-indication-mode) (superword-mode) (tool-bar-mode) (tooltip-mode) (undo-tree-visualizer-selection-mode) (unify-8859-on-decoding-mode) (unify-8859-on-encoding-mode) (url-handler-mode) (use-hard-newlines) (vc-parent-buffer) (view-mode) (visible-mode) (visual-line-mode) (window-divider-mode) (ws-butler-global-mode) (xref-etags-mode)) ```

[greghendershott]

The motivation for starting the back end before the REPL was mainly racket-xp-mode and also multiple REPLs #338 , and is discussed in this blog post.

There is a racket-stop-back-end command. Also racket-start-back-end, which is actually a (re)start (it effectively does a stop if running, then starts). I made those commands "just in case", but didn't document then because I didn't consider them to be thing normal users should actually ever really need. However if you're hacking on Racket Mode's back end, itself, then you might need them.

Does that explain what you're seeing, and what you need?

[greghendershott]

By the way, each REPL still does the Racket thing with a custodian so that state like ports etc. is still reset when you exit the REPL -- and indeed still reset on every racket run. That hasn't changed!

If there is some state that is not covered by the custodian, then let's try to figure that out and make sure that it is.

[greghendershott]

One bit of state that isn't (can't?) be managed by a custodian is Racket's module registry. Which sounds like it might match your description?

Anyway if you racket-stop-back-end it will reset that, too, and automatically exit any/all REPLs.

[greghendershott]

To follow-up:

1. I think the TL;DR is: Use racket-stop-back-end instead of racket-repl-exit if you want to stop the entire Racket Mode back end and all REPLs. AFAICT that should get the same "clean slate" effect that you want.

2. Having said that, if you have some specific scenario where you would like not to need to do this -- not even the old way with racket-repl-exit? Then if you can give me more details (a small example program and steps to follow would be great!) I'd be happy to take a look and try to understand what's going on.

[vkz]

Been muling this over a little. Maybe what I really need isn't necessarily to fix this issue but rather come up with a pleasant enough way to develop "services" and long-running programs in Racket without losing the conveniences REPLs afford and my hair. The issue described above came about because I was watching the filesystem, automatically recompiling any sources that changed and auto-restarting the service. REPL becomes pretty much redundant here but the experience is less than pleasant. I'd like my REPL back please. Starting the system in the REPL is fine and dandy but then everything flies in the face of "we load one module and its namespace - that ought to be good enough for everyone" Racket mantra. I experimented with staring off in REPL and then (enter! mod) which does the trick but I guess goes against the grain of what both DrRacket and racket-mode are about. It also doesn't address the problem of reloading changed dependencies. We are using (require ...) everywhere, not dynamic-rerequire after all.

I wonder if this deserves a separate discussion? How did you do this for Extramaze? Maybe even Racket mailing list but I fear it'll inevitably fall into top-level hopeless routine and not everyone there cares about Emacs experience.

[greghendershott]

I'll mull this over, too.

A few quick thoughts, just in the meantime:

• Good news: After the recent changes, racket-mode (editing) and racket-xp-mode no longer need a REPL, no longer need to "run" your program, in order to do useful things while editing and exploring code. They full-expand the code but don't run it. In that respect things have improved. You no longer need to use the racket-repl-mode part to do IDE-ish activities.

• I think you're already familiar with racket-reloadable? It makes you identify parts of your program that are supposed to be reloadable. On the one hand this is a nuisance, on the other hand it provides clarity. (A probably poor analogy would be how Haskell makes you split out the functional from side-effecting parts of your program?)

• When I've done "system-y" things, I find a lot of the battle is simply avoiding things that initialize themselves at the module level -- e.g. (define the-thread (thread ...)) -- and replacing that explicit start-xxx functions. For one thing, this helps make initialization order explicit in a main.rkt that requires multiple such modules -- now it calls the start-xxx functions explicitly. For another thing, each such module can be evaluated and run in a REPL without the initializations happening, and you can interact with its functions, um, functionally. :) And turn some of those interactions into unit tests.

So again those are just quick thoughts, and possibly Captain Obvious ones, at that. I'd be happy to discuss and consider more.

[greghendershott]

Hi! I've thought about this more, but I'm not sure how to proceed.

I don't (yet) see a specific bug to fix. Maybe I should relabel this from "bug" to "enhancement".

But then also, I don't (yet) understand a specific feature to implement, with a recipe I could use to say if it's working as-desired.

So I think I'm going to go with the "question" label. (Or I could add a "brainstorming" label, I suppose?)

I hope this doesn't seem defensive. I'm definitely still open to discussing, understanding, and considering other workflows and how Racket Mode might (or might not) be able to support those! I think I'd just like that in its own "bucket", separate from to-do items that are ready to be done.

[vkz]

Appears neither of us have a solid idea how to proceed, so I may even suggest that we close it for now instead of relabeling. It's the kind of thing that may linger for years until someone gets annoyed enough to just close it. I have some ideas that may result in actual code or suggestions we'll be able to discuss. I'm working on several projects in Racket atm and trying to keep tabs on what trips me up and my workflow in general.

Thank you for giving it thought!

[acarrico]

Here is an example, but may not be much use. Seems like documentation is the best solution? Starting with something like:

#lang racket/gui

(define frame (new frame% (label "example")))
(send frame show #t)

When you hit F5, you get a new repl with a new frame, shown in X11 (normally). After hacking around a while, I have seen racket-mode in a state where the new frame is not shown in X11. In this state, running the program at the command line does create the frame, and killing the process does allow racket-mode to create a new repl which acts normally.

I don't know how to reproduce this reliably though.

[acarrico]

Actually I can reproduce this behavior by hitting ^d to terminate the repl. C-c C-\ seems to do it also. M-x racket-stop-back-end fixes the issue.

[greghendershott]

@acarrico Thanks I opened #520 as its own issue because I suspect that can be fixed/closed independently --- at least, you have a specific recipe to test. Having said that, it might turn out to shed light on the original, so no worries, of course it's fine/great you started it off by making a comment here.