search

extendr / rextendr

An R package that helps scaffolding extendr-enabled packages or compiling Rust code dynamically
https://extendr.github.io/rextendr/
Other
174 stars 25 forks source link

Failing to run `rextendr::document` #317

Closed albertsgarde closed 9 months ago

albertsgarde commented 9 months ago

The problem

I am trying to follow the guide at https://cran.r-project.org/web/packages/rextendr/vignettes/package.html to build an R package in Rust. I have successfully created a package with usethis::create_package and run rextendr::use_extendr(). The next step is then to run rextendr::document() in the created directory, but this fails for me with the error

r$> rextendr::document()
ℹ Generating extendr wrapper functions for package: rustest.
ℹ Re-compiling rustest (debug build)
── R CMD INSTALL ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────Error in `(function (command = NULL, args = character(), error_on_status = TRUE, …`:
! System command 'Rcmd.exe' failed
---
Exit status: 1
stdout & stderr: <printed>
---
Type .Last.error to see the more details.

.Last.error gives me

r$> .Last.error
<system_command_status_error/rlib_error_3_0/rlib_error/error>
Error in `(function (command = NULL, args = character(), error_on_status = TRUE, …`:
! System command 'Rcmd.exe' failed
---
Exit status: 1
stdout & stderr: <printed>
---
Backtrace:
 1. rextendr::document()
 2. rextendr::register_extendr(path = pkg, quiet = quiet)
 3. pkgbuild::compile_dll(path = path, quiet = quiet, force = compile)
 4. pkgbuild:::withr_with_makevars(compiler_flags(debug), { …
 5. pkgbuild:::withr_with_envvar(c(R_MAKEVARS_USER = makevars_file), { …
 6. base::force(code)
 7. base::force(code)
 8. pkgbuild:::withr_with_envvar(c(DEBUG = "true"), build())
 9. base::force(code)
10. pkgbuild::build()
11. pkgbuild:::install_min(path, dest = install_dir, components = "libs", args = if (needs_clean(path)) "--preclean", …
12. pkgbuild::rcmd_build_tools("INSTALL", c(path, paste("--library=", dest, …
13. pkgbuild::with_build_tools({ …
14. pkgbuild:::withr_with_path(rtools_path(), code)
15. base::force(code)
16. base::withCallingHandlers(callr::rcmd_safe(..., env = env, spinner = FALSE, …
17. callr::rcmd_safe(..., env = env, spinner = FALSE, show = FALSE, …
18. callr:::run_r(options)
19. base::with(options, with_envvar(env, do.call(processx::run, c(list(bin, …
20. base::with.default(options, with_envvar(env, do.call(processx::run, …
21. base::eval(substitute(expr), data, enclos = parent.frame())
22. base::eval(substitute(expr), data, enclos = parent.frame())
23. callr:::with_envvar(env, do.call(processx::run, c(list(bin, args = real_cmdargs, …
24. base::force(code)
25. base::do.call(processx::run, c(list(bin, args = real_cmdargs, stdout_line_callback = real_callback(stdout), …
26. (function (command = NULL, args = character(), error_on_status = TRUE, …
27. base::throw(new_process_error(res, call = sys.call(), echo = echo, …
28. | base::signalCondition(cond)
29. (function (e) …
30. asNamespace("callr")$err$throw(e)

What is the problem? What could I do to solve it?

Details

I am running 64-bit Windows 10. I am using a fresh install of 4.3.1 R

Initially I got the error 

r$> rextendr::document()
✔ Saving changes in the open files.
ℹ Generating extendr wrapper functions for package: rustest.
Error: Could not find tools necessary to compile a package
Call `pkgbuild::check_build_tools(debug = TRUE)` to diagnose the problem.

but GPT-4 helped me to fix it by installing Rtools (version 4.3)

I wanted to build the Rust crate manually (i.e., with cargo) to see if that would help and to do that I needed to set the following environment variables

export R_INCLUDE_DIR="C:/Program Files/R/R-4.3.1/include"
export LIBRSYS_R_VERSION="4.3.1"

but they did not help with the problem mentioned above.

I have also tried adding both the Rtools directory and R home to the PATH but none helped.

CGMossa commented 9 months ago

We'd love to help you along. I'm unsure what is going on. Please run devtools::document() from time to time as well; rextendr::document() doesn't call devtools::document() with everything, you might be interested in.

I'd like to urge you also to run 

pkgbuild::check_build_tools(debug = TRUE)

as described in the error message. Please insert the results here if you're not able to act on it.

Here's another thing entirely: If I run

> R.home("home")
[1] "/usr/lib/R"
> R.home("home") %>% 
  fs::dir_ls()
/usr/lib/R/COPYING      /usr/lib/R/SVN-REVISION 
/usr/lib/R/bin          /usr/lib/R/etc          
/usr/lib/R/lib          /usr/lib/R/library      
/usr/lib/R/modules      /usr/lib/R/site-library

This is R_HOME. I'm using fs::dir_ls() to show you what I've got in there. If I instead use R.home("bin") I get this path

> R.home("bin")
[1] "/usr/lib/R/bin"

And in that path


> R.home("bin") %>% fs::dir_ls()
/usr/lib/R/bin/BATCH         /usr/lib/R/bin/COMPILE       
/usr/lib/R/bin/INSTALL       /usr/lib/R/bin/LINK          
/usr/lib/R/bin/R             /usr/lib/R/bin/REMOVE        
/usr/lib/R/bin/Rcmd          /usr/lib/R/bin/Rd2pdf        
/usr/lib/R/bin/Rdconv        /usr/lib/R/bin/Rdiff         
/usr/lib/R/bin/Rprof         /usr/lib/R/bin/Rscript       
/usr/lib/R/bin/SHLIB         /usr/lib/R/bin/Stangle       
/usr/lib/R/bin/Sweave        /usr/lib/R/bin/build         
/usr/lib/R/bin/check         /usr/lib/R/bin/config        
/usr/lib/R/bin/exec          /usr/lib/R/bin/javareconf    
/usr/lib/R/bin/libtool       /usr/lib/R/bin/mkinstalldirs 
/usr/lib/R/bin/pager         /usr/lib/R/bin/rtags

and please especially notice Rcmd.

On windows you'll get another path, it would b %R_HOME%/bin/x64/, which contains Rcmd, etc.

It is these binaries you should have access to by terminal upon installing R. Somehow, this is not setup for you when you install R normally.

For instance, on Windows, I use scoop, and it makes these binaries available in PATH upon installing. This doesn't mean you should use scoop, I'm just pointing out that this step is a convenience step that is assumed elsewhere.

albertsgarde commented 9 months ago

Running devtools::document() I get

r$> devtools::document()
ℹ Updating rustest documentation
Setting `RoxygenNote` to "7.2.3"
ℹ Loading rustest
ℹ Re-compiling rustest (debug build)
── R CMD INSTALL ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────Error in `(function (command = NULL, args = character(), error_on_status = TRUE, …`:
! System command 'Rcmd.exe' failed
---
Exit status: 1
stdout & stderr: <printed>
---
Type .Last.error to see the more details.

So basically the same as for rextendr:: I had trouble getting knitr to install correctly, but that should be fixed now. Is there some way to get more information about how Rcmd.exe failed? When I run .Last.error as the message suggests I just get the same stack trace as before which doesn't really help me.

Running pkgbuild::check_build_tools(debug = TRUE) I get

r$> pkgbuild::check_build_tools(debug = TRUE)
Found in Rtools 4.3 installation folder
Your system is ready to build packages!

Which is nice I guess.

After adding %R_HOME%/bin/x64/ to PATH I can use both Rterm and Rcmd directly from the command line. Specifically, this means that my terminal can find the Rcmd.exe command which the error message says is failing.

CGMossa commented 9 months ago

Just a small follow-up: If you add something to the PATH, you must shutdown all the apps, and then run them again. Whatever program "helps" launch a new program, inherits the previous program's environment variable. This means you need to start from fresh, to have a program re-take environment variables.

albertsgarde commented 9 months ago

Just to double check that everything was updated to the new PATH I restarted my computer. Rterm still works, but the error messages are also the same.

Ilia-Kosenkov commented 9 months ago

Can you build any other package from source? This does not seem to be directly connected to {rextendr}. What about sharing the whole example so we can run it locally & verify it? Please, share also the output of rextendr::rust_sitrep().

albertsgarde commented 9 months ago

I just tried building a pure R package and I get the same error, so I think you are right that this is not (only) related to rextendr. When I run Rcmd from the command line I get the help page, and if I give it any arguments it does nothing. Looks like there's something wrong with my installation, which is weird because it's a complete fresh install. Maybe I'll try an older version.

Running rextendr::rust_sitrep() I get

> rextendr::rust_sitrep()
Rust infrastructure sitrep:
✔ "rustup": 1.26.0 (5af9b9484 2023-04-05)
✔ "cargo": 1.71.1 (7f1d04c00 2023-07-29)
ℹ host: x86_64-pc-windows-msvc
ℹ toolchain: stable-x86_64-pc-windows-msvc
ℹ targets: wasm32-unknown-unknown, x86_64-pc-windows-msvc, and x86_64-unknown-linux-gnu
Ilia-Kosenkov commented 9 months ago

Good thing we figured this out. BTW your rust setup looks OK, so should be no problem. I think I have not seen issues where no package can be built. Maybe your rtools are in a bad shape? Although I think rtools are not needed to build a pure R package.

Another suggestion would be to just run R as admin. Maybe it hits some weird file permissions issue.

Ilia-Kosenkov commented 9 months ago

Oh, sorry, not OK, you need x86_64-pc-windows-gnu target :)

CGMossa commented 9 months ago

Can you check if you've got an. Renviron file?

Old tools would require you to fill that out. New rtools doesn't. The two configurations interfere.

Otherwise, you should run R CMD build in a terminal. That will give you a better message about why a regular r package ain't building.

albertsgarde commented 9 months ago

I just tried uninstalling R and getting the previous version (4.2.3) and the behaviour seems to be exactly the same.

Where would I look for the Renviron file? I don't see anything like that in any of the generated directories from either rextendr or plain R.

I have tried running R CMD build (and Rcmd build) in a terminal. Nothing happens, I just get back control immediately. Same happens for all arguments I've tried giving R CMD (or Rcmd), except if I give it no arguments in which case I get 

$ R CMD
Usage: R CMD command args

where 'command' is one of:
  INSTALL  Install add-on packages
  REMOVE   Remove add-on packages
  SHLIB    Make a DLL for use with dynload
  BATCH    Run R in batch mode
  build    Build add-on packages
  check    Check add-on packages
  Rprof    Post process R profiling files
  Rdconv   Convert Rd format to various other formats
  Rdiff    difference R output files
  Rd2pdf   Convert Rd format to PDF
  Rd2txt   Convert Rd format to pretty text
  Stangle   Extract S/R code from vignette
  Sweave    Process vignette documentation
  config   Obtain configuration information about R
  open     Open a file via Windows file associations
  texify   Process a latex file

Use
  R CMD command --help
for usage information for each command.

Again, most of the commands do nothing.

Installed the x86_64-pc-windows-gnu target, but I guess that isn't useful until the R issues are fixed:

> rextendr::rust_sitrep()      
Rust infrastructure sitrep:
✔ "rustup": 1.26.0 (5af9b9484 2023-04-05)
✔ "cargo": 1.71.1 (7f1d04c00 2023-07-29)
ℹ host: x86_64-pc-windows-msvc
ℹ toolchain: stable-x86_64-pc-windows-msvc
ℹ targets: wasm32-unknown-unknown, x86_64-pc-windows-gnu, x86_64-pc-windows-msvc, and x86_64-unknown-linux-gnu

Lastly, I do not have RStudio installed, that isn't a problem is it? It's difficult to find guides for creating R packages which don't assume you use RStudio.

CGMossa commented 9 months ago

The usethis package is what I use for all my package needs.

Rstudio isn't important here.

RENVIRON is typically located in HOME directory, as that's also where stuff like Rprofile would be located.

My suspicion is only relevant if you installed rtools and followed instruction before.

Home is either your user directory in windows, or user / documents.

I'm on mobile so all the above instructions are somewhat vague and not well formatted.

albertsgarde commented 9 months ago

I would be open to trying with the usethis package, but I haven't been able to find out how it would be applicable here (except as used in https://cran.r-project.org/web/packages/rextendr/vignettes/package.html to create the package directory, but that part is solved).

I have installed Rtools (version matching the R version), so your suspicion might be relevant. However, I can't find a RENVIRON file or anything like it in either my user directory or in my documents directory. What I can find is a .RData and a .Rhistory in my user directory and another .Rhistory in the documents directory. They could easily be from old R installations which have since been removed. I even used Everything to search for files called "renviron" and got nothing

CGMossa commented 9 months ago

See https://cran.r-project.org/bin/windows/Rtools/rtools40.html

Also run the check and see what rtools directory it points to (the make check in that link).

A convenience that usethis that I was referring to https://usethis.r-lib.org/reference/edit.html

On Sat, 16 Sept 2023, 11.42 Albert Garde, @.***> wrote:

I would be open to trying with the usethis package, but I haven't been able to find out how it would be applicable here (except as used in https://cran.r-project.org/web/packages/rextendr/vignettes/package.html to create the package directory, but that part is solved).

I have installed Rtools (version matching the R version), so your suspicion might be relevant. However, I can't find a RENVIRON file or anything like it in either my user directory or in my documents directory. What I can find is a .RData and a .Rhistory in my user directory and another .Rhistory in the documents directory. They could easily be from old R installations which have since been removed. I even used Everything to search for files called "renviron" and got nothing

— Reply to this email directly, view it on GitHub https://github.com/extendr/rextendr/issues/317#issuecomment-1722190507, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIDVSCSZYIL7RNOMHH54RTX2VYBFANCNFSM6AAAAAA4ZMOJ5Y . You are receiving this because you commented.Message ID: @.***>

albertsgarde commented 9 months ago

The make check succesfully finds the correct Rtools directory, but I think Rtools isn't an issue as long as I can't get a vanilla R package to build.

After running edit_r_environ() I now have an empty `.Renviron in my home directory. Is there something you suggest I put there?

Also, I've created a StackOverflow question about getting Rcmd to work: https://stackoverflow.com/questions/77117183/rcmd-exe-does-nothing

CGMossa commented 9 months ago

I don't know anymore. Last guess is that your PATH variable is somehow messed up.

I can only say that you can probably ask us (me) more about this in the Discord. Invite here: https://discord.gg/q2eGKHZJ https://discord.gg/q2eGKHZJ

Otherwise, as this issue is not related to rextendr or extendr, I'd like to close it.