Custom R prompts don't work in ESS

[chrisvacc]

I usually have the following in my .RProfile: options(prompt="» ", continue="… ") to customize the prompts like so:

» function(
…   )

but it doesn't work inside ESS.

I had to switch it to:

if(Sys.getenv("INSIDE_EMACS") == ""){
    options(prompt="» ", continue="…  ")
}

Because it seemed to be causing issues.

Also is there a way to nix that little arrow after commands:

[vspinu]

Standard prompt is essential to how ESS works, you shouldn't change it.

The little arrow indicates the start of last command. When you send multiple commands from a script or load a file it is often difficult to tell were the output starts.

[chrisvacc]

Custom prompts work fine in my other ESS-like applications.

Nvim-R is a plugin exactly like ESS, just for vim. You can send multiple lines everything, and I have no issue with either:

(wait sorry... i had to disable them because if the ESS thing.. hold on.. lemme edit)

Ignore this now my .RProfile isn't loading at all, despite it being in the right directory. Great. Another issues to deal with.

Anyway, after posting this I found a pdf from ESS saying this was a known bug and you're supposed to be able to change the custom prompts. since the .RProfile way didn't work they suggested adding this to your ess-custom.el file. The file where people put their customizations.

     (inferior-ess-secondary-prompt         . "» ")
     (inferior-ess-primary-prompt           . "…  "))

But that didn't work.

[chrisvacc]

Wait, here it is from ESS's site. The other one I found was in PDF format.

Changes to the continutation prompt in R (e.g. options(continue = " ")) are not automatically detected by ESS. Hence, for now, the best thing is not to change the continuation prompt. If you do change the continuation prompt, you will need to change accordingly the value of inferior-ess-secondary-prompt in R-customize-alist

Maybe ess-custom.el wasn't the right file? I believe it said "R-customize-alist"

[jabranham]

Like @vspinu said, modifying the prompt is not supported. If you REALLY want to, you can try it though.

To do so, modify inferior-ess-secondary-prompt and inferior-ess-primary-prompt in S-common-cust-alist. I just tried changing them to the characters you're using and nothing broke catastrophically, which was a little surprising :-)

[chrisvacc]

Ha! is it really that bad? that the custom prompts break things.

What file is that in, ess-custom.el? or is it another

[jabranham]

@chrisvacc You can use M-x find-variable S-common-cust-alist to find it easily. It's in ess-s-lang.el.

Some things like prompt navigation (C-c C-n/p) are broken. I'm sure there's other things that are broken as well.

[chrisvacc]

Alright so I put up top:

  '((ess-language                  . "S")
    (inferior-ess-secondary-prompt . "» ")
    (inferior-ess-primary-prompt   . "…  ")
    (inferior-ess-exit-command     . "q()\n")
    (inferior-ess-language-start   . (eval inferior-S-language-start))
    (comint-use-prompt-regexp      . t)  ;;use fields if nil
    (comint-process-echoes     . t)
    ;; these prompt are the same for all S-languages As long as custom prompt
    ;; ends in inferior-ess-primary-prompt everything should work as expected.
    (inferior-ess-primary-prompt   . "> ")
    ;; (inferior-ess-secondary-prompt . "[+:] ") ;; catch Selection: and alike
    (inferior-ess-secondary-prompt . "+ ") ;; catch Selection: and alike
    (comment-start                . "#")
    (comment-add                  . 1)
    (comment-start-skip           . "#+ *")
    (comment-use-syntax           . t)  ; see log for bug report 2013-06-07
    (comment-column               . 40)
    (ess-no-skip-regexp           . (concat "^ *@\\|" (default-value 'ess-no-skip-regexp)))
    ;; inferior-ess-prompt is used by comint for navigation, only if
    ;; comint-use-prompt-regexp is t; (transcript-mode also relies on this regexp)
    (inferior-ess-prompt           . inferior-S-prompt)
    (ess-getwd-command          . "getwd()\n")
    (ess-setwd-command          . "setwd('%s')\n")
    (ess-funargs-command        . ".ess_funargs(\"%s\")\n")
    (fill-nobreak-predicate     . 'ess-inside-string-p)
    (ess-execute-screen-options-command . "options(width=%d, length=99999)\n")
    (font-lock-defaults           . '(ess-build-font-lock-keywords
                                      nil nil ((?\. . "w") (?\_ . "w")))))

Let's see how it goes..

[chrisvacc]

@chrisvacc You can use M-x find-variable S-common-cust-alist to find it easily. It's in ess-s-lang.el.

Some things like prompt navigation (C-c C-n/p) are broken. I'm sure there's other things that are broken as well.

Crap, the one I edited above wasn't the right file. That's why i prefer to just edit source... you have any idea what file I may have edited? I'm somewhat new to emacs (moving from vim) and now I can't figure out what file i put that crap in.

[jabranham]

Nope, but it's easy enough to re-download ess.

Like I said above, trying to use a custom prompt is just asking for trouble :-)

[chrisvacc]

Nah it wasn't the prompt, if you look at my edit history where i posted the code 2 posts ago, my dumbass copied and pasted it from the other file and forgot to remove the second closing parenthesis

so:

'((ess-language                  . "S")
    (inferior-ess-secondary-prompt . "» ")
    (inferior-ess-primary-prompt   . "…  ")) ; <-------- this
    (inferior-ess-exit-command     . "q()\n")

rather than

'((ess-language                  . "S")
    (inferior-ess-secondary-prompt . "» ")
    (inferior-ess-primary-prompt   . "…  ")
    (inferior-ess-exit-command     . "q()\n")

that plus I think it returned the wrong file anyway.

[lionel-]

Did we consider adding a sentinel constant to inputs sent to the inferior? E.g. if user sends

foo()
+bar
baz

We'd send

foo(); "____ESS_EOI____"
+bar; "____ESS_EOI____"
baz; "___ESS_EOI___"

When we catch [1] "___ESS_EOI___" at the console, we filter it out and catch the prompt. This wouldn't work if output is sunk, but that would only happen in a browse() session where the prompt is consistently Browse[n]> anyway.

[lionel-]

hum, I guess this scheme still requires detection of complete user inputs, so wouldn't be an improvement.

Perhaps the best way to deal with this is through ESSR. We could parse(text = ) the input, and deparse it right away. Then we have complete expressions (or an error that we can handle appropriately), and its trivial to append the sentinel.

[chrisvacc]

Also, perhaps there's a way to do it through emacs' prettify symbols mode.

[jabranham]

@lionel- how would that work when users send incomplete lines though?

1 + 
  1

call ess-eval-line-and-step on the first line would be an error if we immediately tried to parse(), right? Or am I missing something?

[chrisvacc]

@lionel- how would that work when users send incomplete lines though?

1 + 
  1

call ess-eval-line-and-step on the first line would be an error if we immediately tried to parse(), right? Or am I missing something?

Hey, I don't know that much about the technical aspects of ESS, but I do know that in R they handle that with options(continue="+ ") so it pops up as

> 1 + 
+ 1

Not sure if that's helpful or obvious, but hopefully it's helpful.

[lionel-]

@jabranham Would it make sense to detect "Unexpected end of input" errors from the parser, and wait for more input in that case?

[lionel-]

I guess not, since incomplete inputs wouldn't be evaluated right away.

@vspinu Do we have documentation somewhere of the technical trade-offs and challenges of inferior evaluation?

[lionel-]

We may be able to get into base R a lenient parse() variant that returns a list of two elements, the first with the expressions successfully parsed, and the second with a string containing the last incomplete expression, or NULL if none. That'd be easy to implement and might prove very useful to us.

[mmaechler]

Perhaps the best way to deal with this is through ESSR. We could parse(text = ) the input, and deparse it right away. ...

I hope we don't go there, please. The REPL should really be done by R rather than ESS. If we do it in ESS (in addition, or instead of R), that typically has all kind of low-level side effects, e.g., the NAMED may be increased -- if you use .Internal(inspect(<obj>)) -- and other such side effect which would make ESS+R behave quite differently [[on a very low level, not for 99% of useR code]] and would render ESS quite unuseful for R Core members or other technical R programmers who'd like to inspect lazy evaluation and similar such subtleties. Yes, I think Rstudio does such things all the time.. and indeed it did behave quite different to "bare R" (also R in batch jobs) when such subtleties matter. Smart assignments such as x[.] <- value which did not copy x in newer versions of R, suddently did make copies of x in Rstudio. I really like ESS to remain the R UI of choice for R experts...

[lionel-]

would render ESS quite unuseful for R Core members or other technical R programmers

There are trade-offs. I think it's ok for this small set of users to occasionally run R in a terminal for checking low-level stuff, if it makes ESS otherwise more robust. The current evaluation mechanism is brittle, we need a new approach.

That said, parsing inputs should not have any effects on NAMEDness of user objects. The only side-effects will be on the string pool, symbol pool, and gc. These are pretty harmless. And the evaluation itself would occur pretty much the same way as what is done currently.

Even if it turns out that this incurs side-effects on refcounts, we could do it in a separate process that we'd maintain for this purpose and reuse for all buffers. I think that would be ok because this pre-parsing wouldn't depend on any state, such as loaded or installed packages, and the parsing done in this process should be instantaneous so we could wait on it and avoid concurrency issues.

Note that the existing mechanisms for auto-completion etc are much more side-effectful, they can load packages, change the dispatch path, etc.

[lionel-]

One side-effect of the sentinel idea: it would leak into .Last.value (unless we only use it between expressions).

Just noticed that auto-completion stuff also leaks into .Last.value though.

[vspinu]

Do we have documentation somewhere of the technical trade-offs and challenges of inferior evaluation?

No, there are too many subtleties and backward UI compatibility complexities.

I always thought that the way forward it is to be able to detect complete input on ESS side. We need that functionality in a lot of places anyhow. So basically, for script evaluation we wouldn't use accumulation on the R side at all, just detect full expressions and send them to the inferior. The interaction at the REPL side would stay pure as it is right now. I think this is a reasonable trade-off and would simplify a great deal of UI and code.

Detecting incomplete input on R side is unlikely a good direction. We would still need to be able to accumulate incomplete input and would probably need to communicate back to emacs the state of the evaluation.

In any case this discussion is only remotely related to the OP issue. The OP cannot be solved at the moment because ess-command and a lot of stuff in the process filter rely on standard prompts. Even with standard prompts we have had time tracking them, so allowing arbitrary prompts is not an option. But we could enforce our own special prompt which could potentially eliminate false positives in prompt detection. I haven't had time to investigate this direction yet.

[lionel-]

I always thought that the way forward it is to be able to detect complete input on ESS side.

Agreed. However, relying on the R parser itself would be a guarantee of robustness and accuracy. I think the evaluation side in ESS should be rock solid.

Detecting incomplete input on R side is unlikely a good direction. We would still need to be able to accumulate incomplete input and would probably need to communicate back to emacs the state of the evaluation.

It seems to me that the suggestions above might give us all the info we need to achieve all of this. I might be missing something.

a lot of stuff in the process filter rely on standard prompts

If we have rock-solid detection of output boundaries and prompts, then we could decorate it with text properties. Then we don't need the brittle regexp searches, and we allow arbitrary prompts, which I think is a desirable goal for ESS.

Anyway, just food for thought.

[jabranham]

So lemme double-check my understanding:

with a file like:

1 +
  1

if a user did ess-eval-line-and-step on the first line, we'd send to some ess-specific R process (perhaps one just spawned for this reason) parse(text="1+"). Since that errored, we'd cache 1 +. Then on the next ess-eval-<whatever>, we'd prepend that? So if they did ess-eval-line-and-step on the second line, we'd check parse(text="1 +\n 1). Since that doesn't error, then we'd send that all at once to the actual inferior process they're using?

Seems reasonable to me.

[lionel-]

I think that's not sufficient because the user might send:

1
2 +

which would be a parse error even though there's a complete expression in there. We need to send that expression to the inferior before getting the next inputs. That's why I think we'd need a parse variants in R core that gives more information, as outlined in https://github.com/emacs-ess/ESS/issues/831#issuecomment-461743647. Perhaps there's a way to solve this with existing facilities that I'm not seeing?

[jabranham]

If the whole expression fails, we could check subexpressions by splitting (and then concatenating) on \n or ;. I could imagine that becoming slow on large buffers, but haven't actually tried.

[lionel-]

oh hmm yes perhaps that will work!

I could imagine that becoming slow on large buffers, but haven't actually tried.

That's probably ok because sourcing a buffer or a function would still be done in one piece. If I'm not mistaken we only need this mechanism for other modes of evaluation which should normally have small(ish) inputs.

[vspinu]

It seems to me that the suggestions above might give us all the info we need to achieve all of this. I might be missing something.

If you could state a clear issue and elaborate a bit on how it can be solved with this new proposal, then we could discuss it in a separate thread. The current discussion is too high level. I am not even sure what "problem" we are trying to solve ATM.

we'd prepend that?

Why would we bother with managing accumulation if we know where the input ends? You just can send the entire complete input on the first ess-eval-.

Using an extra R process for boundary detection might work, but I think we can do it just as fine on ESS side, We don't need the full parser, just a reliable top-level sexp detection.

[lionel-]

If you could state a clear issue and elaborate a bit on how it can be solved with this new proposal, then we could discuss it in a separate thread. The current discussion is too high level.

This is all true. However it's also ok to discuss things as they come up, and maybe emerge new ideas, the pre-requisite to fleshed out proposals.

[chrisvacc]

Hey everyone... A somewhat unrelated question. I'm relatively new to ESS (moving from a custom vim-based IDE) and was wondering the best way to learn ESS, or if there was somewhere I can direct questions. For instance IRC, Gitter, email etc. I just have some minor questions about operation. Thanks.

[jabranham]

There's the ess-help listserv (I believe the address is on our website). I'm also on IRC from time to time.

On Tue, Feb 12, 2019, 11:48 AM chrisvacc <notifications@github.com wrote:

Hey everyone... A somewhat unrelated question. I'm relatively new to ESS (moving from a custom vim-based IDE) and was wondering the best way to learn ESS, or if there was somehwere I can direct questions. For instance IRC, Gitter, email etc. I just have some minor questions about operation. Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/emacs-ess/ESS/issues/831#issuecomment-462862113, or mute the thread https://github.com/notifications/unsubscribe-auth/ALXMiQyfci_J6YXENGG48-iB_4DOXQDYks5vMv59gaJpZM4agaMM .

[chrisvacc]

There's the ess-help listserv (I believe the address is on our website). I'm also on IRC from time to time. … On Tue, Feb 12, 2019, 11:48 AM chrisvacc @.*** wrote: Hey everyone... A somewhat unrelated question. I'm relatively new to ESS (moving from a custom vim-based IDE) and was wondering the best way to learn ESS, or if there was somehwere I can direct questions. For instance IRC, Gitter, email etc. I just have some minor questions about operation. Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#831 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ALXMiQyfci_J6YXENGG48-iB_4DOXQDYks5vMv59gaJpZM4agaMM .

Okay I joined. Just noe of the eval shortcuts work. They work when I hit M-x ess-eval, but the shortcuts dont's appear bound to anything.

Could it be because I'm usig doom emacs?