Cannot auto complete when the file contains error

[NOBLES5E]

If I open a racket source code file containing error, then the whole completion function will not work with an error message like this:

Company: backend company-capf error "Process racket-command not running" with args (candidates hash-re)

Personally I think we should not assume every file we want to edit is runnable.

[greghendershott]

Two things here:

1. It's intentional that you can't complete until the file is racket-run, because:

   • To determine the completion candidates we need to use namespace-mapped-symbols.
   • To use that we need to have a namespace.
   • To get that we need to e.g. use module->namespace on the file.
   • That's effectively racket-run.

   I agree this is annoying. It annoys me! Unfortunately I don't have a better answer. Could racket-mode provide some "default" completion candidates? Sure, but what should they be? The symbols from #lang racket? #lang racket/base? #lang typed/racket? What if the file #lang unusual? So...I don't know. I'm open to suggestions/ideas!

2. To communicate that it's intentional, racket-mode gives the message, Completions not available until you ‘racket-run’ this buffer. I'm not sure why you're not seeing that. Maybe because you're using an older racket-mode, or, maybe it's related to company-mode? I'll investigate that.

[greghendershott]

Also, when company-mode is involved, so is its "show docs" feature, for which racket-mode uses its racket-describe feature. For that to work properly, it needs to know which identifier is meant -- from which module. For example is it the define from racket or typed/racket. To do this properly means having a true define identifier from a Racket namespace, not just the text "define". Which again means some equivalent of racket-run-ing the file.

[NOBLES5E]

Thanks for your reply Greg! From your response it seems that we have this problem because the completion candidates are given by actually running the code instead of using a static analysis tool?

It seems to me this is what most REPLs do (e.g. IPython, the completion candidates change as we run more statements). This makes sense because in a REPL we run statements one by one and we make sure the previous lines are run correctly. Most IDEs (also some modes in emacs, e..g. anaconda mode for Python) use static analysis tools (e.g. jedi for Python) to perform this task. In this way errors can be tolerated.

I just did a quick search and did not see any mature static analysis lib for racket. Maybe that's the reason why we have to run the code to get completions?

How about the following suggestion: when we start racket-mode, the repl only reads the first line specifying the language and provides basic completion for that language. We can then send the parts of the file we want to run to the repl manually and interactively to give the repl more and more information (and only the information we want to give the repl). (I think this is some other modes in emacs do, e.g. cider mode for Clojure if I remember correctly). In this way we can manually avoid sending code containing error to the repl and we have the full control of what we want to complete. Does this make sense to you?

---

Edit: That is to say maybe we can decouple the process of start/connection to the repl and the process of running code in the repl. If we do want to run the whole file in the repl we can manually trigger something like send buffer to repl after the connection to the repl.

[greghendershott]

Thank you for the suggestion!

Although there is a #lang racket/load, which is effectively the same as, "just type these expressions at the REPL one by one", it's rarely used. "The top-level is hopeless". We use modules.

A file foo.rkt starting with #lang language reads as (module foo language expressions ...).

If instead of evaluating that module expression, we evaluate expressions one-by-one at a REPL -- it often won't have the same result.

As just one example, within a module form, it is OK to have forward references (you can use a binding before it is defined; you're not obligated to declare as in Clojure). There are more examples; see "hopeless".

(And that's just for "vanilla" #lang racket. For langs that provide an interesting #%module-begin -- that can party on expressions -- it really won't work.)

So I think not possible to change racket-run this way.

---

But. I totally understand the complaint re completions. I myself am annoyed when I'm editing a file that failed to run due to an error. I'm as likely to want completion to work, then. Maybe more so!

So maybe there is some strategy to try, but only if/after there was an error. Maybe we could try to do what you describe, with the module language and requires, at least until that maybe also fails, and use whatever we've accumulated for completions. Would that be satisfactory, or, backfire somehow (e.g. the completions differ in unexpected ways)? I'm not sure.

[NOBLES5E]

Thanks! I support making what I suggested as a backup method when there is an error (and we can display a message in the minibuffer if that causes confusion). It seems to me that we will not sacrifice anything in this way.

It seems that currently using racket-run, after we add more definitions in the file, the completion candidates for these new definitions are not shown until I actually send them to the repl. Is this effectively something like what I described? If so, does this imply when we add more definitions, we should call racket-run again instead of just sending the added parts to the repl?

[greghendershott]

The intended workflow is similar to DrRacket:

1. You racket-run run the file.
2. You can explore the results in the REPL. For instance, try calling a function that you defined in the source file.
3. You can also use the usual Emacs lispy commands to send just fragments of the file to the REPL to be evaluated. This is merely a shortcut for copy and paste.
4. The REPL is merely a transient playground. The next time you racket-run, the state of the REPL is completely reset to the result of evaluating the file. Nothing you did only in the REPL survives. And that's considered to be a good thing.

In other words, ground truth is your source file (hopefully as managed by git).

(Admittedly this is in contrast to a strand of lisp heritage where you poke mutable state in the REPL... and hope you can recreate it later. Some thoughts about that here. I'm not saying that style is wrong. I am saying racket-mode isn't attempting to support that.)

---

So, I think it's a given for racket-mode that completions will be available only after racket-run.

The interesting question is what we were discussing above: What if racket-run encounters some Racket syntax error? Could it try to evaluate as much else of the file as possible, to at least get some useful completions? Probably it could.

I'm worried about files having top-level expressions like (push-the-red-button!). If the user chooses racket-run, they know those will be executed. So that's OK. The part I'm worried about is, could (push-the-red-button!) misbehave based on some prior error-filled expressions not having been evaluated -- as part of this fallback effort to "run as much as possible for the sake of completions".

It might be safe to limit this to the module language and require forms. Caveats:

• That's only likely to work well for #lang racket flavor langs, where the symbol require actually expands to the actual require identifier. That's probably acceptable.
• It would mean that definitions in the file aren't available for completion. But maybe dabbrev would suffice for that?

[greghendershott]

As an update, I've been working on this. It turned out to be trickier than I expected. Or at least, slower (for me) to figure out than I expected.

I think I'm getting close? But I want to dog-food it for awhile.

Some WIP on this branch.

[NOBLES5E]

Thanks! I saw your commit and that is actually better than what I expected! I previously thought we can just read the #lang line and give a repl for that language :)

[greghendershott]

Yesterday I thought this was ready and squashed to a single commit on a new issue-272 branch in preparation for merging.

But this morning I noticed it showed the error message twice, when the error is in a file required by the file you're running. So another commit to fix that.

I'll live with it more, to see if I notice anything else.

Will likely re-squash before merging.