search

extendr / rextendr

An R package that helps scaffolding extendr-enabled packages or compiling Rust code dynamically
https://extendr.github.io/rextendr/
Other
184 stars 27 forks source link

Unable to build example #15

Closed Ilia-Kosenkov closed 3 years ago

Ilia-Kosenkov commented 3 years ago

I installed latest github version of the package and tried creating the following rust function from the README:

rust_function("fn add(a:f64, b:f64) -> f64 { a + b }")

I unfortuantely experienced a liking issue at the end, see attached log:

```pwsh build directory: C:\Users\[redacted]\AppData\Local\Temp\Rtmp04R3wM\file46582e2c1e4d Updating git repository `https://github.com/extendr/extendr` Updating crates.io index Compiling winapi-build v0.1.1 Compiling winapi v0.3.9 Compiling winapi v0.2.8 Compiling proc-macro2 v1.0.24 Compiling unicode-xid v0.2.1 Compiling syn v1.0.58 Compiling extendr-engine v0.1.11 (https://github.com/extendr/extendr#f714309f) Compiling lazy_static v1.4.0 Compiling kernel32-sys v0.2.2 Compiling quote v1.0.8 Compiling extendr-macros v0.1.11 (https://github.com/extendr/extendr#f714309f) Compiling libR-sys v0.2.1 Compiling extendr-api v0.1.11 (https://github.com/extendr/extendr#f714309f) Compiling rextendr1 v0.0.1 (C:\Users\[redacted]\AppData\Local\Temp\Rtmp04R3wM\file46582e2c1e4d) error: linking with `link.exe` failed: exit code: 1181 | = note: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.28.29333\\bin\\HostX64\\x64\\link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.1109b7oki3iy8t63.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.147sjo61quy01gmx.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.18c95s8sfqsxquuj.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.1ibpcj3kfkp6aui2.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.1lqos2f8068mw4qh.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.26e1qwh59i1q5m7k.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.28pcu9knbknhvzkw.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2c9dqcehnk29gg5h.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2j9zzh0cm8tm34dr.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2lh12q8e2e0bj40g.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2mquew07qtassp2j.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.2zt21rjm9icixcp2.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.328vlxktswt53wtg.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.33qkn1pkb117k5tg.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.3e9fnqq3jbqxzfk9.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.3hkoh4ve50ze28sj.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.3q25qttx9oje29z4.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.3rg0lshqq2jwm9lu.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.446tnh1hgeolf4fm.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.46z1eys4ytheu3u7.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.4fdhf75i23bk8vxa.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.4x2o391x3w6z5ml2.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.4zaytgzpph72bfw6.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.59upnde7k2408fk1.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.60phqctaujov6k4.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.br35nq041eolkor.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.f9qxssc7l1rmtgz.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.meaiqnde68um5p1.rcgu.o" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.nykpq4yd0pxi0xe.rcgu.o" "/OUT:C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.dll" "/DEF:C:\\Users\\[redacted]\\AppData\\Local\\Temp\\rustcu1SLss\\lib.def" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.45yv15xwvrrq7lsi.rcgu.o" "/OPT:REF,NOICF" "/DLL" "/IMPLIB:C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps\\rextendr1.dll.lib" "/DEBUG" "/NATVIS:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis" "/LIBPATH:C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d/target\\debug\\deps" "/LIBPATH:C:/PROGRA~1/R/R-40~1.3\\bin\\x64" "/LIBPATH:C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d\\target\\debug\\deps\\libextendr_api-e53357031a4dd514.rlib" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d\\target\\debug\\deps\\libextendr_engine-5644b2bd52a06f57.rlib" "C:\\Users\\[redacted]\\AppData\\Local\\Temp\\Rtmp04R3wM\\file46582e2c1e4d\\target\\debug\\deps\\liblibR_sys-f9732ac30d4d11f1.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd-f771f8d0374ceeb1.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libpanic_unwind-9049f341d2774a65.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_demangle-6a47dea777db971c.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libhashbrown-a3b06009982e3d9c.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_alloc-ebb710aea302de49.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunwind-4304b0c1525af3c3.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcfg_if-c70f620fc91130c5.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liblibc-3b06f844280b3802.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liballoc-86edfc7ab798fc86.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_core-3aa1c50c964075c2.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcore-3ec787c945bbba26.rlib" "C:\\Users\\[redacted]\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-937aede5bfe8081e.rlib" "R.lib" "advapi32.lib" "ws2_32.lib" "userenv.lib" "msvcrt.lib" = note: LINK : fatal error LNK1181: cannot open input file 'R.lib' error: aborting due to previous error error: could not compile `rextendr1` To learn more, run the command again with --verbose. Error: Rust code could not be compiled successfully. Aborting. Execution halted ```

This happened at least on Windows. One of the problems I see is that the linking is attempted agains R.lib, while it should be R.dll. It seems that either Makevars or something else is poorly configured.

With that said, maybe we should have 'real' tests attached to this repository? For instance, creating such add function and checking its output, because the package can be installed and testthat succeeds, yet I am unable to use it.

Ilia-Kosenkov commented 3 years ago

Just checked it in WSL2 - works fine on Ubuntu, so looks like again some Windows issue.

Ilia-Kosenkov commented 3 years ago

Also, if I attempt to re-build bindings using rextendr:::install_libR_bindings() produces the following error:

```pwsh Updating crates.io index Compiling winapi-build v0.1.1 Compiling winapi v0.3.9 Compiling winapi v0.2.8 Compiling kernel32-sys v0.2.2 Compiling libR-sys v0.2.1 Compiling build-libR-sys v0.0.1 (C:\Users\[redacted]\AppData\Local\Temp\RtmpY7Zm9D\file42d8675047f9) Finished dev [unoptimized + debuginfo] target(s) in 27.03s Error in normalizePath(path.expand(path), winslash, mustWork) : path[1]="C:/Users/[redacted]/R/win-library/4.0/rextendr/rust/libR-sys/Cargo.toml": The system cannot find the path specified Calls: -> -> normalizePath Execution halted build directory: C:\Users\[redacted]\AppData\Local\Temp\RtmpY7Zm9D\file42d8675047f9 ```

I believe it originates from here (or is somehow related to)

https://github.com/extendr/rextendr/blob/9332feed61b42fdd896f79a6ca5d5faccba26846/R/install_libR_bindings.R#L81

yutannihilation commented 3 years ago

Curious. R.lib seems handled here. Is .Platform$dynlib.ext .lib on Windows?

https://github.com/extendr/rextendr/blob/4ec4d4413de0fe8b22b5d09ccfa41da90a647af0/R/source.R#L146-L150 https://github.com/extendr/rextendr/blob/4ec4d4413de0fe8b22b5d09ccfa41da90a647af0/R/source.R#L189-L198

Ilia-Kosenkov commented 3 years ago

@yutannihilation, Nope, it says .dll.

I would say it arises somewhere here https://github.com/extendr/rextendr/blob/4ec4d4413de0fe8b22b5d09ccfa41da90a647af0/R/source.R#L121-L132 When building the resulting library, cargo grabs incorrect R.lib and ultimately fails.

yutannihilation commented 3 years ago

Ah, sorry, you are right. But it succeeds on my Windows. My guess is that you need to use Rtools' GCC, not MSVC this time because what R provides is probably for Rtools (I don't have MSVC toolchain installed).

``` > rust_function("fn add(a:f64, b:f64) -> f64 { a + b }") build directory: C:\Users\...\AppData\Local\Temp\RtmpUfCcCt\file11a48f04315 Updating git repository `https://github.com/extendr/extendr` Updating crates.io index Downloading crates ... Downloaded libR-sys v0.2.1 Downloaded quote v1.0.8 Downloaded syn v1.0.58 Compiling winapi-x86_64-pc-windows-gnu v0.4.0 Compiling winapi-build v0.1.1 Compiling winapi v0.3.9 Compiling winapi v0.2.8 Compiling proc-macro2 v1.0.24 Compiling unicode-xid v0.2.1 Compiling syn v1.0.58 Compiling extendr-engine v0.1.11 (https://github.com/extendr/extendr#f714309f) Compiling lazy_static v1.4.0 Compiling kernel32-sys v0.2.2 Compiling quote v1.0.8 Compiling extendr-macros v0.1.11 (https://github.com/extendr/extendr#f714309f) Compiling libR-sys v0.2.1 Compiling extendr-api v0.1.11 (https://github.com/extendr/extendr#f714309f) Compiling rextendr1 v0.0.1 (C:\Users\...\AppData\Local\Temp\RtmpUfCcCt\file11a48f04315) Finished dev [unoptimized + debuginfo] target(s) in 3m 24s > add(2.5, 4.7) [1] 7.2 ```
Ilia-Kosenkov commented 3 years ago

@yutannihilation, Ok, this worked for me. But I have two issues with this:

  1. This forced me to set *-gnu toolchain as default, which is a somewhat system-wide modification compared to the default toolchain used to build rust projects under Windows. It is OK to ask user to install something extra but I would prefer to not modify any system defaults.
  2. I believe we can build correct library either using --target with default msvc toolchain or by providing +toolchain flag to cargo to override default toolchain.

I can try investigating this.

yutannihilation commented 3 years ago

Makes sense. I agree with you.

clauswilke commented 3 years ago

Just a general comment on tests: I agree there needs to be a test suite. It just must not run on CRAN, as it will cause never ending problems otherwise. I wanted to write some code that simply doesn't call any of the tests on CRAN, rather than add skip_on_cran() everywhere, but I did't get to it and that's where it stopped.

There also shouldn't be any examples that require a working Rust install and that aren't wrapped into \dontrun{}.

I'm happy to receive PRs that add tests, given these constraints.