rmheiberger commented 5 years ago

summary defun R is not redefined to find a specific version of the R executable when ess-rterm-version-paths has been run.

I assume that is because R is no longer a defun, it is now a defalias

These completions are done after ess-site and before M-x R

"emacs 25.1 rmh.lnk" ;; my usual Windows setup which includes (require 'ess-site) and therefore gets c:/Program Files (x86)/GNU Emacs 25.1/share/emacs/site-lisp/ess/ess-site.elc which is Vince's setup using ess-16.10

M-x R Possible completions are: R R-3.2.3-32bit R-3.2.3-64bit R-3.2.4revised-32bit R-3.2.4revised-64bit R-3.3.0-32bit R-3.3.0-64bit R-3.3.1-32bit R-3.3.1-64bit R-3.3.2-32bit R-3.3.2-64bit R-3.3.3-32bit R-3.3.3-64bit R-3.4.0patched-32bit R-3.4.0patched-64bit R-3.4.1-32bit R-3.4.1-64bit R-3.4.3patched-32bit R-3.4.3patched-64bit R-3.4.3patched20180205-32bit R-3.4.3patched20180205-64bit R-3.5.0-32bit R-3.5.0-64bit R-devel-32bit R-devel-64bit R-fix-T-F R-initialize-on-start R-mode R-newest R-site-search R-transcript-mode Rd-describe-major-mode Rd-font Rd-mode Rd-mode-calculate-indent Rd-mode-indent-line Rd-mode-insert-item Rd-mode-insert-section Rd-mode-insert-skeleton Rd-preview-help Rd-submit-bug-report Rnw-mode

emacs -Q followed by manually loading c:/Users/rmh.DESKTOP-60G4CCO/Downloads/ess-18.10.2/lisp/ess-site.el This download is the latest releasse from ess.r-project.org

M-x R Possible completions are: R-3.2.3-32bit R-3.2.3-64bit R-3.2.4revised-32bit R-3.2.4revised-64bit R-3.3.0-32bit R-3.3.0-64bit R-3.3.1-32bit R-3.3.1-64bit R-3.3.2-32bit R-3.3.2-64bit R-3.3.3-32bit R-3.3.3-64bit R-3.4.0patched-32bit R-3.4.0patched-64bit R-3.4.1-32bit R-3.4.1-64bit R-3.4.3patched-32bit R-3.4.3patched-64bit R-3.4.3patched20180205-32bit R-3.4.3patched20180205-64bit R-3.5.0-32bit R-3.5.0-64bit R-devel-32bit R-devel-64bit R-initialize-on-start R-mode R-newest R-transcript-mode

Notice that R is missing in 18.10.2.

Indeed, there is no occurrence of 'defun R ' in 18.10.2. That string does appear in 16.10 in file ess-r-d.el

We do see in 18.10.2

(symbol-function 'R) run-ess-r

and run-ess-r is defined in file ess-r-mode.el

16.10 R and 18.10.2 run-ess-r have essentially the same definition, with some cl name changes and with capitalization of "R" in variable names.

18.10.2 also has (defalias 'R #'run-ess-r)

16.10 M-x R ESS (R, R (default)) starting data directory? ~/ and then R is opened with running R.

And here is the critical piece: immediately after opening emacs (which loads ess-site) R has been redefined to the same type of function as R-newest. It is full of hex and won't copy cleanly. Here is the non-hex (symbol-function 'R)

[(&optional start-args) %% lots of hex here

[ess-dialect start-args current-prefix-arg ess-microsoft-p system-type ess-R-readline ess-write-to-dribble-buffer format " (R): ess-dialect=%s, buf=%s, start-arg=%s current-prefix-arg=%s " cygwin "--ess " "--no-readline " ...] 7 ("c:/Program Files (x86)/GNU Emacs 25.1/share/emacs/site-lisp/ess/ess-r-d.elc" . 11937) "P"]

18.10.2 R starting project directory? ~/ Searching for program: No such file or directory, Rterm R appears with no process

The Messages window shows apply: Searching for program: No such file or directory, Rterm

immediately after opening emacs (which loads ess-site) R still has the value of the run-ess-r function (symbol-function 'R) (lambda (&optional start-args) "Call 'R', the 'GNU S' system from the R Foundation. Optional prefix (\[universal-argument]) allows to set command line arguments, such as .....

jabranham commented 5 years ago

It's worth noting that this is the same behavior on GNU/Linux systems too. If R isn't found in exec-path, then we fail to start it. It's less of an issue there since it's .... well, a better operating system. PATH gets setup correctly and whatnot. You can fix the behavior on windows by setting inferior-ess-r-program to point to Rterm in your init file.

M-x R-newest works just fine on windows, right? We could alias R to run-ess-r on non-windows systems and to R-newest on windows systems.

Alternatively, we could just assume that PATH is set correctly, that users modify exec-path, or they set inferior-ess-r-program to point to Rterm directly. This is probably the best long-term solution, since none of the currently active ESS developers have a windows machine.

rmheiberger commented 5 years ago

I just tried (defalias 'R #'run-ess-r) on Windows. It seems to work.

Be aware of a potential timing issue? Do you have to wait for R-newest to be defined (after the search for all versions) before you do the defalias?

On Windows you absolutely do not want to touch PATH. Nor do you want to touch inferior-ess-r-program. Nor do you want to ask the user to either of those. The solution ess-rterm-arch-version was invented for this purpose and is still the best solution.

If no currently active ess developer is using windows, then the only prudent thing to do is to make sure the external behavior of ESS does not change.

Background: The convention for using PATH on Windows is not the same as the convention on Unix. Only directories that you specifically want are in the PATH. Anything else needs the full pathname.

On Mon, Dec 10, 2018 at 5:52 PM Alex Branham notifications@github.com wrote:

It's worth noting that this is the same behavior on GNU/Linux systems too. If R isn't found in exec-path, then we fail to start it. It's less of an issue there since it's .... well, a better operating system. PATH gets setup correctly and whatnot. You can fix the behavior on windows by setting inferior-ess-r-program to point to Rterm in your init file.

M-x R-newest works just fine on windows, right? We could alias R to run-ess-r on non-windows systems and to R-newest on windows systems.

Alternatively, we could just assume that PATH is set correctly, that users modify exec-path, or they set inferior-ess-r-program to point to Rterm directly. This is probably the best long-term solution, since none of the currently active ESS developers have a windows machine.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/emacs-ess/ESS/issues/787#issuecomment-446004965, or mute the thread https://github.com/notifications/unsubscribe-auth/AB_co91wg0isfSyI8ntSmgGQJ6G96_urks5u3uWggaJpZM4ZMRaZ .

lionel- commented 5 years ago

On Windows you absolutely do not want to touch PATH.

Are you sure? As far as I know, users are supposed to add Rtools to PATH if they want to compile packages on Windows. Why couldn't they do the same for R, to run it from ESS?

mmaechler commented 5 years ago

They never ever had to. PATH on Windows is not a concept used much. Rtools is for experts anyway, they know about the PATH. 99% of the windows users don't.

lionel- commented 5 years ago

Agreed that it should work out of the box. I was intrigued about the notion that PATH should absolutely not be touched on Windows.

lionel- commented 5 years ago

@rmheiberger It seems unlikely that the R symbol was redefined in 16.10. It is more likely something to do with the default value of inferior-ess-r-program after the cleanup of ess-site. IIUC we should take the result of (ess-find-newest-R) as a default for inferior-ess-r-program.

@jabranham would it make sense to set the defcustom to nil (at least on Windows, instead of Rterm, perhaps on other platforms as well) and inject a default with (or inferior-ess-r-program (ess-find-newest-R))?

jabranham commented 5 years ago

Is there an accepted way of starting subprocesses on Windows? In other words, does anyone know how M-x run-python or M-x run-scheme or any of the other REPLs work?

lionel- commented 5 years ago

Python just look for the command in this custom:

(defcustom python-shell-interpreter "python"
  "Default Python interpreter for shell."
  :type 'string
  :group 'python)

So in this case you either set a path manually or add python to your PATH I guess. I think scanning probable locations of R installations to find a reasonable default is the right thing to do though.

vspinu commented 5 years ago

How about checking if inferior-ess-r-program is pointing to a valid executable and fallback on ess-find-newest-R?

Is there an accepted way of starting subprocesses on Windows?

Same as on other platforms. Emacs call-process should handle those transparently. Or what's your concern here?

jabranham commented 5 years ago

@vspinu like this?

diff --git a/lisp/ess-r-mode.el b/lisp/ess-r-mode.el
index 0fee6a2d..3aa18a61 100644
--- a/lisp/ess-r-mode.el
+++ b/lisp/ess-r-mode.el
@@ -252,6 +252,11 @@ When t, loading a file into a namespaced will output information
 about which objects are exported and which stay hidden in the
 namespace.")

+(defvar ess-newest-R nil
+  "Stores the newest version of R that has been found.
+Used as a cache, within `ess-find-newest-R'. Do not use this value
+directly, but instead call the function \\[ess-find-newest-R].")
+
 (defun ess-r-font-lock-syntactic-face-function (state)
   (let ((string-end (save-excursion
                       (and (nth 3 state)
@@ -476,6 +481,9 @@ will be prompted to enter arguments interactively."
    (format
     "\n(R): ess-dialect=%s, buf=%s, start-arg=%s\n current-prefix-arg=%s\n"
     ess-dialect (current-buffer) start-args current-prefix-arg))
+  (unless (executable-find inferior-ess-r-program)
+    (setq inferior-ess-r-program (or ess-newest-R
+                                     (ess-find-newest-R))))
   (let* ((r-always-arg
           (if (or ess-microsoft-p (eq system-type 'cygwin))
               "--ess "
@@ -808,12 +816,6 @@ as `ess-r-created-runners' upon ESS initialization."
 (define-obsolete-function-alias
   'ess-r-versions-create 'ess-r-define-runners "ESS 18.10")

-(defvar ess-newest-R nil
-  "Stores the newest version of R that has been found.
-Used as a cache, within `ess-find-newest-R'. Do not use this value
-directly, but instead call the function \\[ess-find-newest-R].")
-
-
 (defcustom ess-prefer-higher-bit t
   "Non-nil means prefer higher bit architectures of R.
 e.g. prefer 64 bit over 32 bit.  This is currently used only

@lionel- thanks for looking that up. So IIUC M-x julia is broken on windows as well since we don't do this whole directory-search thing for it. I still can't believe that OS doesn't provide a way to start a process without specifying the full path.

lionel- commented 5 years ago

I still can't believe that OS doesn't provide a way to start a process without specifying the full path.

It does have PATH just like unixes, but you need to change its value in the Windows registry, which is very scary for most users ;)

lionel- commented 5 years ago

I think the problem is more the lack of a common binary directory like /usr/bin/ or /usr/local/bin. The R installer on macOS adds a link from /usr/bin/R to the location in the application framework, which makes it work by default in any shell.

jabranham commented 5 years ago

What's the permissions structure like in these ProgramFiles environment variable locations, do you know? Are we introducing a security issue by searching for any executable named "Rterm"?

I'm trying to understand why other modes don't do this sort of thing.

rmheiberger commented 5 years ago

Thank you Martin for reminding us that most Windows users are mostly not expert computer users.

Screed on importance of Windows

here is some prevalence information https://en.wikipedia.org/wiki/Usage_share_of_operating_systems#Desktop_and_laptop_computers

Desktop/Laptop operating system browsing statistics Windows 81.76% macOS 13.49% Unknown 1.99% Linux 1.68% Chrome OS 1.08% Desktop OS market share according to StatCounter for September 2018.[54] Chrome OS is also based on the Linux kernel.

This is from http://cran-logs.rstudio.com/ http://cran-logs.rstudio.com/2018/2018-12-01-r.csv.gz It contains all downloads of R from http://cran.rstudio.com/ on December 1.

table(R_2018_12_01[,5])

osx src win 871 106 2094

And the winner is ... Windows.

comments on ess-rterm-versions

The user documentation for ess-r-versions and ess-rterm-versions is in ess-18.10.2/doc/ess.pdf at the bottom of page 52.

I am comparing the definition of ess-rterm-version-paths in 16.10 and 18.10.2. In 16.10 it was in ess-site.el inside an "(if ess-microsoft-p" condition. In 18.10.2 it is in ess-r-mode.el and seems to be run for all operating systems.

@lionel Most Windows users click on an icon. The command window is rarely used. Icons are set by the program installation procedure and include the full pathname. --"I still can't believe that OS doesn't provide a way to start a process without specifying the full path." Yes Windows does. it provides the icon.
@lionel The R symbol was redefined for Windows in 16.10 and earlier. R is initially defined in ess-r-d.el with the function definition that now appears as run-ess-r in ess-r-mode.el. My first message in this thread showed the value of R after ess-site is run.
@jabranham "or they set inferior-ess-r-program to point to Rterm directly." That is essentially what we (ESS) were doing, but inside a "(let" and under program control.

This is from 16.10 (symbol-function 'R-3.5.0-64bit) (lambda (&optional start-args) "Call the R version 'R-3.5.0-64bit' using ESS. This function was generated by ess-r-versions-create." (interactive "P") (let ((inferior-R-version "R-3.5.0-64bit") (inferior-R-program-name "c:/PROGRA~1/R/R-35~1.0/bin/x64/Rterm.exe")) (R start-args))) This is from 18.10.2 (symbol-function 'R-3.5.0-64bit) (lambda (&rest --cl-rest--) "Start this process version in an inferior ESS buffer. Function defined using ess-define-runner." (interactive "P") (apply (quote (lambda ... ...)) (quote --path--) (quote --dialect--) (quote --name--) --cl-rest--)) The 18.10.2 version is less easy to read than the older version. From looking at the definition of ess-define-runner, it has some "(let*" statements, so it is probably doing the same thing.

@vspinu I think you are supporting the existing procedure of finding all occurences of Rterm and defining an R-x.y.z-nbit function.
@jabranham "I'm trying to understand why other modes don't do this sort of thing." My guess is that most other programs assume that only the current version is on a computer. R from the beginning has placed each version of R in its own directory in a parallel position to other versions of R. ESS designed M-x R to find the most recent, and gives alternate expressions for finding older versions.

permissions emacs ls on Windows is not helpful. cygwin ls is helpful drwx------+ 1 4294967295 mkpasswd 0 Mar 22 2018 doc/ I attempted to write a new file in that directory and it wouldn't let me.

security I don't think we are introducing anything. We don't look for all executables, only those in the standard locations that the R installation uses, or if the user changes that location and tells us, then where the user tells us to look.

New worry. In my first message of the thread, I noted that in 18.10.2 "R" itself was missing from R <tab> That strikes me as misbehavior. Is that because R is a defalias instead of a defun? Looking now, I realize that Rd-* is also missing. Is that a problem?
summary I believe that redefining R to be the same as R-newest is the right solution. I think R should appear as an option in R <tab>.

lionel- commented 5 years ago

@lionel Most Windows users click on an icon. The command window is rarely used. Icons are set by the program installation procedure and include the full pathname. --"I still can't believe that OS doesn't provide a way to start a process without specifying the full path." Yes Windows does. it provides the icon.

Richard, the above is not a quote from me. I'll answer anyway that macOS and even Linux users also use icons to launch applications.

The R symbol was redefined for Windows in 16.10 and earlier.

I don't think this is accurate. I have read the 16.10 code this morning and again tonight, and can't find such a redefinition. The output of symbol-function you pasted in your first message appears to be the output of the original (defun 'R ...) which never gets redefined. The fact that it is now an alias to run-ess-r shouldn't have any influence. I think the cause of the current behaviour is that ess-site.el used to define inferior-ess-r-program to point to one of the automatically detected paths.

The 18.10.2 version is less easy to read than the older version.

This generated function is not meant to be read. If you compile it you'll even see bytecode in there. On the other hand the code to generate it is much easier to read and maintain thanks to Alex's efforts.

I believe that redefining R to be the same as R-newest is the right solution.

That doesn't seem right to me. Where would the original definition live? We should use the path of the newest R detected as a fallback for inferior-ess-r-program. As far as I can see, that is how 16.10 works, so I think we are on the same page.

In 18.10.2 "R" itself was missing from R

Hmm... I see R as the first autocompletion command. However I use helm-M-x, could that have an influence?

nverno commented 5 years ago

Back when I used Windows I had a whole battery of .bat / .cmd / .ps1 scripts to set the PATH variables before launching applications. For example, if you install your R versions in an unusual location you can write an emacs.bat script that sets the path with a weird R location and launch emacs from that shell. Then you can still just double click the icon and have a locally modified PATH that doesn't change your registry.

I even went as far as creating a database of all the executables in C: root, but at that point I realized Linux did this by default anyway and I was wasting time. But, the point is you can set the PATH without altering your registry quite easily to include non-standard locations when running applications, eg. emacs.

For example, I found this old script for adding an R directory, after which you could call emacs and have a modified PATH. It's hideous, but such is life with dos files. If I remember correctly, cygwin and MSYS are full of similar shims.

@echo off
:: Add latest R version to path
set rdir="C:\Program Files\R"
set major=0
set minor=0
set release=0

setlocal EnableDelayedExpansion
for /f %%x in ('dir /b %rdir%') do (
    set "prog=%%x"
    set "prog=!prog:R-=!"
    for /f "tokens=1-3 delims=." %%i in ("!prog!") do (
        if %%k GTR !release! set release=%%k
    if %%j GTR !minor! (
        set minor=%%j
        set release=%%k
    )
    if %%i GTR !major! (
        set major=%%i
        set minor=%%j
        set release=%%k
    )
    )
)

set latest=%rdir:"=%\R-%major%.%minor%.%release%\bin\x64

:: Add latest version to PATH
@echo Adding %latest% to path
endlocal & set PATH=%latest%;%PATH%

rmheiberger commented 5 years ago

@lionel- sorry. that was a quote from jabranham addressed TO you.

What is helm-M-x?

16.10 does not have the string inferior-ess-r-program.

"That doesn't seem right to me. Where would the original definition live? " the original definition stays in run-ess-r.

rmheiberger commented 5 years ago

@nverno your script looks like variation of the search we do beginning in line (setq ess-rterm-version-paths in file ess-r-mode.el

emacs-ess / ESS

M-x R in 18.10.2 does not work on Windows #787

[(&optional start-args) %% lots of hex here