Current buffer has no process

[jthaman]

On Windows 10 / Emacs 27.1, 26.3 / ESS 20210126, I occasionally edit files on network drives, and run into some odd ESS behavior. It's hard to pin down the issue, but after working interactively with R scripts on network drives, the iESS buffer loses connection, and shows no process in the modeline.

In the messages buffer, I can only find the phase user-error: Current buffer has no process. If I try to continue to interact with the broken process, I get the messageError running timer ‘ess--idle-timer-function’: (wrong-type-argument stringp nil)

When this happens I have to restart the R process, but I inevitably run into it again.

The ESS buffer shows the output

(R): ess-dialect=R, buf=JTH/v3_data_read.R, start-arg=nil
 current-prefix-arg=nil
(inferior-ess: waiting for process to start (before hook)
(inferior-ess 3): waiting for process after hook(R): inferior-ess-language-start=options(STERM='iESS', str.dendrogram.last="'", editor='emacsclient', show.error.locations=TRUE)
 (ess-search-list ... ) after 'search()
', point-max=294
(nil): created new alist of length 13

I've only run into this no process issue while working interactively with R scripts on network drives under Windows 10, but I'm not sure that's part of the problem.

Here is my complete ESS configuration:

(use-package ess
  :defer t
  :ensure t
  :hook
  (ess-help-mode . evil-normal-state)
  :custom
  (inferior-ess-fix-misaligned-output t)
  (ess-eldoc-show-on-symbol t)
  (ess-gen-proc-buffer-name-function 'ess-gen-proc-buffer-name:projectile-or-directory)
  (ess-eval-visibly 'nil "Don't hog Emacs")
  (ess-style 'RStudio)
  (ess-use-flymake nil "Syntax checking is usually not helpful")
  (ess-tab-complete-in-script nil "Do not interfere with Company")
  (ess-use-ido nil "Prefer Ivy/Counsel")
  (ess-history-directory (expand-file-name "ESS-history/" no-littering-var-directory))
  (inferior-R-args "--no-save")
  (ess-R-font-lock-keywords
   (quote
    ((ess-R-fl-keyword:keywords . t)
     (ess-R-fl-keyword:constants . t)
     (ess-R-fl-keyword:modifiers . t)
     (ess-R-fl-keyword:fun-defs . t)
     (ess-R-fl-keyword:assign-ops . t)
     (ess-R-fl-keyword:%op% . t)
     (ess-fl-keyword:fun-calls . t)
     (ess-fl-keyword:numbers . t)
     (ess-fl-keyword:operators)
     (ess-fl-keyword:delimiters)
     (ess-fl-keyword:=)
     (ess-R-fl-keyword:F&T))))
  (ess-ask-for-ess-directory nil)
  (ess-smart-S-assign-key nil)
  (ess-indent-with-fancy-comments nil)
  :config
  (defun my/add-pipe ()
    "Add a pipe operator %>% at the end of the current line.
Don't add one if the end of line already has one.  Ensure one
space to the left and start a newline with indentation."
    (interactive)
    (end-of-line)
    (unless (looking-back "%>%" nil)
      (just-one-space 1)
      (insert "%>%"))))

Maybe this is related to #1091?

[lionel-]

If you manage to find a reproducible procedure for triggering that bug, can you please post a trace? You can generate one with M-x ess-elisp-trace-mode. Given the amount of output, this likely won't be practical unless you can systematically trigger the bug though.

Another way that is less informative but also less verbose and intrusive is to set (setq ess-verbose t). You can then find a log in the *ESS* buffer.

[jthaman]

Hi Lionel, I can sort of reproduce the bug, at least on my setup.

1. Start Emacs 27.1 on Windows 10
2. Navigate to R script on network drive
3. Start ESS and start running some code chunks (ess-eval-region-or-function-or-paragraph-and-step)
4. After a few chunks have been executed, switch to iESS buffer and type library(stats). The process will disconnect while typing this command.

ESS dribble is at https://gist.githubusercontent.com/jthaman/c800d0554ba50ca44dfa8f782576ee63/raw/7c8cd9c497d11afd8c5fb6f8bf8c0c682373a9a8/gistfile1.txt

Tracebug output (separate session from the dribble, but same steps to reproduce): https://gist.githubusercontent.com/jthaman/4bd101183700d45a1d000f12738435ec/raw/a8630d5a3290db5e57b5aaa6939ea01b635b48a0/tracebug-output.txt

[vspinu]

I think I know where this comes from. On the timeout-interrupt (#1104) the process is somehow stuck in the temporary buffer and not returned back to the original buffer. It doesn't happen always, only sometimes. This is why it's so difficult to debug.

When this happens, and the temporary buffer is still there, try C-c C-z. You will be brought to the temp buffer because the ESS process association is using processes, not buffers.

[plpxsk]

I'm also experiencing similar behavior.

@jthaman check that your remote ESS process initializes correctly. For me, it seems it times out, due to the issue linked by @vspinu

Specifically, I load up remote R, then (ess-remote) and the initialization times out:

ess-command: Timeout during background ESS command ‘{##' Source FILE into an environment. ...

I'm trying to increase timeout but I am unsuccessful. Currently trying something like:

(ess-command (ess-remote "*shell*" "R") :timeout 10)

``` Using process ‘RR-pred’ Loading ESSR into remote ... Using process ‘RR-pred’ ess-command: Timeout during background ESS command ‘{##' Source FILE into an environment. After having a look at each new object in##' the environment, decide what to do with it. Handles plain objects,##' functions, existing S3 methods, S4 classes and methods.##' @param fallback_env environment to assign objects which don't exist in the##' package namespace.ess.ns_source <- function(file, visibly, output, expr, package = "", verbose = FALSE, fake.source = FALSE, fallback_env = NULL, local_env = NULL) { pname <- paste("package:", package, sep = "") envpkg <- tryCatch(as.environment(pname), error = function(cond) NULL) if (is.null(envpkg)) { if (suppressWarnings(require(package, quietly = TRUE, character.only = TRUE))) { envpkg <- tryCatch(as.environment(pname), error = function(cond) NULL) } else { ## no such package; source in current (local) user environment return(.ess.source(file, visibly = visibly, output = output, local = local_env, fake.source = fake.source)) } } envns <- tryCatch(asNamespace(package), error = function(cond) NULL) if (is.null(envns)) stop(gettextf("Can't find a namespace environment corresponding to package name '%s\"", package), domain = NA) ## Here we know that both envns and envpkg exists and are environments if (is.null(fallback_env)) fallback_env <- .ess.ns_insert_essenv(envns) ## Get all Imports envs where we propagate objects pkgEnvNames <- Filter(.ess.is_package, search()) packages <- lapply(pkgEnvNames, function(envName) substring(envName, 9)) importsEnvs <- lapply(packages, function(pkgName) parent.env(asNamespace(pkgName))) ## Evaluate the FILE into new ENV env <- .ess.ns_evalSource(file, visibly, output, substitute(expr), package, fake.source) envPackage <- getPackageName(env, FALSE) if (nzchar(envPackage) && envPackage != package) warning(gettextf("Supplied package, %s, differs from package inferred from source, %s", sQuote(package), sQuote(envPackage)), domain = NA) ## Get all sourced objects, methods and classes allObjects <- objects(envir = env, all.names = TRUE) allObjects <- allObjects[!(allObjects %in% c(".cacheOnAssign", ".packageName"))] MetaPattern <- methods:::.TableMetaPattern() ClassPattern <- methods:::.ClassMetaPattern() allPlainObjects <- allObjects[!(grepl(MetaPattern, allObjects) | grepl(ClassPattern, allObjects))] allMethodTables <- allObjects[grepl(MetaPattern, allObjects)] allClassDefs <- allObjects[grepl(ClassPattern, allObjects)] ## PLAIN OBJECTS and FUNCTIONS: funcNs <- funcPkg <- newFunc <- newNs <- newObjects <- newPkg <- objectsNs <- objectsPkg <- character() dependentPkgs <- list() for (this in allPlainObjects) { thisEnv <- get(this, envir = env) thisNs <- NULL ## NS if (exists(this, envir = envns, inherits = FALSE)){ thisNs <- get(this, envir = envns) if(is.function(thisNs) || is.function(thisEnv)){ if(is.function(thisNs) && is.function(thisEnv)){ if(.ess.differs(thisEnv, thisNs)){ environment(thisEnv) <- environment(thisNs) .ess.assign(this, thisEnv, envns) funcNs <- c(funcNs, this) if(exists(".__S3MethodsTable__.", envir = envns, inherits = FALSE)){ S3_table <- get(".__S3MethodsTable__.", envir = envns) if(exists(this, envir = S3_table, inherits = FALSE)) .ess.assign(this, thisEnv, S3_table) } } }else{ newNs <- c(newNs, this) } }else{ if(!identical(thisEnv, thisNs)){ .ess.assign(this, thisEnv, envns) objectsNs <- c(objectsNs, this) } } }else{ newNs <- c(newNs, this) } ## PKG if (exists(this, envir = envpkg, inherits = FALSE)){ thisPkg <- get(this, envir = envpkg) if(is.function(thisPkg) || is.function(thisEnv)){ if(is.function(thisPkg) && is.function(thisEnv)){ if(.ess.differs(thisPkg, thisEnv)){ environment(thisEnv) <- environment(thisPkg) .ess.assign(this, thisEnv, envpkg) funcPkg <- c(funcPkg, this) } }else{ newPkg <- c(newPkg, this) } }else{ if(!identical(thisPkg, thisEnv)){ .ess.assign(this, thisEnv, envpkg) objectsPkg <- c(objectsPkg, this) } } }else{ newPkg <- c(newPkg, this) } if (!is.null(thisNs)) { isDependent <- .ess.ns_propagate(thisEnv, this, importsEnvs) newDeps <- stats::setNames(list(packages[isDependent]), this) dependentPkgs <- c(dependentPkgs, newDeps) } } ## deal with new plain objects and functions for (this in intersect(newPkg, newNs)) { thisEnv <- get(this, envir = env, inherits = FALSE) if (exists(this, envir = fallback_env, inherits = FALSE)){ thisGl <- get(this, envir = fallback_env) if (.ess.differs(thisEnv, thisGl)) { if (is.function(thisEnv)) { environment(thisEnv) <- envns newFunc <- c(newFunc, this) } else { newObjects <- c(newObjects, this) } .ess.assign(this, thisEnv, fallback_env) if (.is.essenv(fallback_env)) .ess.assign(this, thisEnv, .GlobalEnv) } } else { if (is.function(thisEnv)) { environment(thisEnv) <- envns newFunc <- c(newFunc, this) } else { newObjects <- c(newObjects, this) } .ess.assign(this, thisEnv, fallback_env) if (.is.essenv(fallback_env)) .ess.assign(this, thisEnv, .GlobalEnv) } } if(length(funcNs)) objectsNs <- c(objectsNs, sprintf("FUN[%s]", paste(funcNs, collapse = ", "))) if(length(funcPkg)) objectsPkg <- c(objectsPkg, sprintf("FUN[%s]", paste(funcPkg, collapse = ", "))) if(length(newFunc)) newObjects <- c(newObjects, sprintf("FUN[%s]", paste(newFunc, collapse = ", "))) ## CLASSES classesPkg <- classesNs <- newClasses <- character() for(this in allClassDefs){ newPkg <- newNs <- FALSE thisEnv <- get(this, envir = env) if(exists(this, envir = envpkg, inherits = FALSE)){ if(!.ess.identicalClass(thisEnv, get(this, envir = envpkg))){ .ess.assign(this, thisEnv, envir = envpkg) classesPkg <- c(classesPkg, this) } }else{ newPkg <- TRUE } if(exists(this, envir = envns, inherits = FALSE)){ if(!.ess.identicalClass(thisEnv, get(this, envir = envns))){ .ess.assign(this, thisEnv, envir = envns) classesNs <- c(classesNs, this) } }else{ newNs <- TRUE } if(newNs && newPkg){ if(exists(this, envir = fallback_env, inherits = FALSE)){ if(!.ess.identicalClass(thisEnv, get(this, envir = fallback_env))){ .ess.assign(this, thisEnv, envir = fallback_env) newClasses <- c(newClasses, this) } }else{ .ess.assign(this, thisEnv, envir = fallback_env) newClasses <- c(newClasses, this) } } } if(length(classesPkg)) objectsPkg <- gettextf("CLS[%s]", sub(ClassPattern, "", paste(classesPkg, collapse = ", "))) if(length(classesNs)) objectsNs <- gettextf("CLS[%s]", sub(ClassPattern, "", paste(classesNs, collapse = ", "))) if(length(newClasses)) newObjects <- gettextf("CLS[%s]", sub(ClassPattern, "", paste(newClasses, collapse = ", "))) ## METHODS: ## Method internals: For efficiency reasons setMethod() caches ## method definition into a global table which you can get with ## 'getMethodsForDispatch' function, and when a method is dispatched that ## table is used. When ess-developer is used to source method definitions the ## two copies of the functions are identical up to the environment. The ## environment of the cached object has namespace:foo as it's parent but the ## environment of the object in local table is precisely namspace:foo. This ## does not cause any difference in evaluation. methodNames <- allMethodTables methods <- sub(methods:::.TableMetaPrefix(), "", methodNames) methods <- sub(":.*", "", methods) methodsNs <- newMethods <- character() for (i in seq_along(methods)){ table <- methodNames[[i]] tableEnv <- get(table, envir = env) if(exists(table, envir = envns, inherits = FALSE)){ inserted <- .ess.ns_insertMethods(tableEnv, get(table, envir = envns), envns) if(length(inserted)) methodsNs <- c(methodsNs, gettextf("%s{%s}", methods[[i]], paste(inserted, collapse = ", "))) }else if(exists(table, envir = fallback_env, inherits = FALSE)){ inserted <- .ess.ns_insertMethods(tableEnv, get(table, envir = fallback_env), envns) if(length(inserted)) newMethods <- c(newMethods, gettextf("%s{%s}", methods[[i]], paste(inserted, collapse = ", "))) }else{ .ess.assign(table, tableEnv, envir = fallback_env) newMethods <- c(newMethods, gettextf("%s{%s}", methods[[i]], paste(objects(envir = tableEnv, all.names = T), collapse = ", "))) } } if(length(methodsNs)) objectsNs <- c(objectsNs, gettextf("METH[%s]", paste(methodsNs, collapse = ", "))) if(length(newMethods)) newObjects <- c(newObjects, gettextf("METH[%s]", paste(newMethods, collapse = ", "))) if (verbose) { msgs <- unlist(list( if(length(objectsPkg)) sprintf("PKG: %s", paste(objectsPkg, collapse = ", ")), if(length(objectsNs)) sprintf("NS: %s", paste(objectsNs, collapse = ", ")), if(length(dependentPkgs)) .ess.ns_format_deps(dependentPkgs), if(length(newObjects)) { env_name <- .ess.ns_env_name(fallback_env) sprintf("%s: %s", env_name, paste(newObjects, collapse = ", ")) })) if(length(msgs)) .ess_mpi_message(paste(msgs, collapse = " ")) } invisible(env)}}’ ```

[jthaman]

@pavopax Paul, I may look into that.

[jthaman]

I've now run into the bug on Emacs 26.3, and updated the issue. I don't think Emacs version explains the variance in the bug anymore.

edit: The bug is also occurring while working on local drives.

[lionel-]

Thanks for reporting everyone. I've merged an attempt at fixing this (#1106). Can you please try it once melpa has updated and see if that fixes it for you please?

If you're no longer seeing this issue, can you please also try with this:

(setq ess--command-default-timeout 1)

This will cause the timeout errors you've been seeing with remotes (we've bumped the default timeout to 30s in the mean time). It'd be helpful if you could confirm that the process is not lost, i.e. you don't see "Current buffer has no process" after a timeout error.

[jthaman]

Thanks so much for this work, Lionel. I'm seeing some improvements, but will continue to test with my R scripts.

I am finding there is now some internal ESS output that gets dumped into the iESS process. For example, I'm seeing some output like

> 
(list "base" '(("..." . "") ("na.rm" . "FALSE")) '("..." "na.rm"))
ess-output-delimiter79-END
> > 
(list "base" '(("object" . "") ("..." . "") ("digits" . "") ("quantile.type" . "7")) '("object" "..." "intercept" "split" "expand.split" "keep.zero.df" "all" "full" "maxsum" "digits" "quantile.type" "dispersion" "correlation" "symbolic.cor" "test" "tol" "ny" "names" "loadings" "cutoff" "max_frames" "dir" "srcrefs" "useSource"))
ess-output-delimiter80-END

interlaced with my R output.

[lionel-]

Do you see anything relevant in *messages*? Timeout errors?

[jthaman]

Nothing in *messages*, and so far nothing notable in *ESS*, but I will keep testing. I have not encountered a timeout error yet, but I am encountering some brief flashes of the title bar.

It's unclear to me, but the Windows 10 titlebar repeatedly flashes when I encounter the 'current buffer has no process' error.

[lionel-]

Good news, I can reproduce your hardship by inserting this snippet in .ess_funargs() (which is used on the R side by eldoc):

    withCallingHandlers(
        interrupt = function(...) Sys.sleep(2),
        Sys.sleep(2)
    )

The calling handler simulates a long interrupt. This will be helpful to create unit tests that simulate a slow responding process.

With this I see the same behaviour as you just described @jthaman. I also tried it with a version of ESS anterior to #1106 and I could reproduce the process loss. So the PR did fix that particular problem. To fix the remaining issue:

1. I now realise that we can't block on process interruptions so I'm going to look into interrupting asynchronously.

2. I also wonder if we could detect slow remotes to automatically disable background commands. This will avoid getting the process busy while a background command is interrupting asynchronously.

[lionel-]

@jthaman By the way in the meantime you can set ess-can-eval-in-background to nil when you're working with remotes, that should solve the issue by disabling completions.

[jthaman]

Nice, I just tested. Setting ess-can-eval-in-background to nil disables the output. Also makes eldoc go away though :(

[plpxsk]

Reporting a related bug:

I've been getting a bunch of these new Disabling output delimiter because CMD failed to parse errors that have been introduced in #1108 referenced above.

More importantly, the remote ESS process is freezing up my whole Emacs, and C-g helps, but it takes a few seconds of frantic cancellation. This is even without sending commands - just trying to navigate my R (Rmd with polymode) buffer, and things hang!

The issue goes away with: "Setting ess-can-eval-in-background to nil disables the output" per above.

In addition, I am still getting timeouts when first loading ESSR process. Messages:

Loading ESSR into remote ...
ess-command: Timeout during background ESS command ‘{
##' Source FILE into an environment. After having a look at each new object in
##' the environment, decide what to do with it. Handles plain objects,
##' functions, existing S3 methods, S4 classes and methods.
##' @param fallback_env environment to assign objects which don't exist in the
##'     package namespace
.ess.ns_source <- function(file, visibly, output, expr,
                           package = "", verbose = FALSE,
                           fake.source = FALSE,
                           fallback_env = NULL,
                           local_env = NULL) {
...

This is on the latest available ESS:

M-x ess-version:
ess-version: 18.10.3snapshot [elpa: 20210217.935] (loaded from

and the timeout is 30s, as expected in this new version:

C-h v ess--command-default-timeout
ess--command-default-timeout is a variable defined in ‘ess-inf.el’.
Its value is 30

[vspinu]

Should be reported as a new bug. This one is already closed and it's not the same issue.