search

oracle / fastr

A high-performance implementation of the R programming language, built on GraalVM.
Other
623 stars 64 forks source link

Error installing jsonlite package on macOS Catalina with Fastr 20.3.0 #178

Open hsselman opened 3 years ago

hsselman commented 3 years ago

I tried installing jsonlite using a local package from source, attached here 00_jsonlite_1.6.1.tar.gz. The same result was achieved using default mirror in FastR. I was using GraalVM + FastR 20.3.0, same error on latest 21.0.0.2. Trying to install on macOS Catalina 10.15.7.

Error installing jsonlite:

> install.package('jsonlite')
* installing *source* package ‘jsonlite’ ...
** package ‘jsonlite’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c base64.c -o base64.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c collapse_array.c -o collapse_array.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c collapse_object.c -o collapse_object.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c collapse_pretty.c -o collapse_pretty.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c escape_chars.c -o escape_chars.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c integer64_to_na.c -o integer64_to_na.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c is_datelist.c -o is_datelist.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c is_recordlist.c -o is_recordlist.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c is_scalarlist.c -o is_scalarlist.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c modp_numtoa.c -o modp_numtoa.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c null_to_na.c -o null_to_na.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c num_to_char.c -o num_to_char.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c parse.c -o parse.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c prettify.c -o prettify.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c push_parser.c -o push_parser.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c r-base64.c -o r-base64.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c register.c -o register.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c row_collapse.c -o row_collapse.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c transpose_list.c -o transpose_list.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c validate.c -o validate.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c yajl/yajl.c -o yajl/yajl.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c yajl/yajl_alloc.c -o yajl/yajl_alloc.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c yajl/yajl_buf.c -o yajl/yajl_buf.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c yajl/yajl_encode.c -o yajl/yajl_encode.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c yajl/yajl_gen.c -o yajl/yajl_gen.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c yajl/yajl_lex.c -o yajl/yajl_lex.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c yajl/yajl_parser.c -o yajl/yajl_parser.o
clang -I".../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fPIC  -O2  -DFASTR -DNO_GNUR  -c yajl/yajl_tree.c -o yajl/yajl_tree.o
.../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/bin/safe-forward-ar rcs yajl/libstatyajl.a yajl/yajl.o yajl/yajl_alloc.o yajl/yajl_buf.o yajl/yajl_encode.o yajl/yajl_gen.o yajl/yajl_lex.o yajl/yajl_parser.o yajl/yajl_tree.o
clang -dynamiclib -Wl,-headerpad_max_install_names,-rpath,.../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/lib/,-undefined,dynamic_lookup -L.../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/lib -undefined dynamic_lookup -single_module -multiply_defined suppress -Wl,-rpath,.../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/lib/,-undefined,dynamic_lookup -L.../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/lib -o jsonlite.so base64.o collapse_array.o collapse_object.o collapse_pretty.o escape_chars.o integer64_to_na.o is_datelist.o is_recordlist.o is_scalarlist.o modp_numtoa.o null_to_na.o num_to_char.o parse.o prettify.o push_parser.o r-base64.o register.o row_collapse.o transpose_list.o validate.o -Lyajl -lstatyajl
installing to .../R/x86_64-apple-darwin-library/fastr-20.3.0-3.6/00LOCK-jsonlite/00new/jsonlite/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘jsonlite’ in FUN(X[[i]], ...):
 no such symbol C_collapse_array in package /Users/haroldselman/R/x86_64-apple-darwin-library/fastr-20.3.0-3.6/00LOCK-jsonlite/00new/jsonlite/libs/jsonlite.so
Error: loading failed
> bin/R --vm.version 
openjdk version "11.0.9" 2020-10-20
OpenJDK Runtime Environment GraalVM CE 20.3.0 (build 11.0.9+10-jvmci-20.3-b06)
OpenJDK 64-Bit Server VM GraalVM CE 20.3.0 (build 11.0.9+10-jvmci-20.3-b06, mixed mode, sharing)

Output of R built-in function sessionInfo().

> sessionInfo()
FastR version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin19.6.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   .../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/lib/libRblas.dylib
LAPACK: .../GraalVMWithFastR/20.3.0/graalvm-ce-java11-20.3.0/Contents/Home/languages/R/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] tools_3.6.1

Adding --no-test-load makes installation work, but loadint the package library(jsonlite) you still get an error: 

> library(jsonlite)
Error: package or namespace load failed for ‘jsonlite’ in library.dynam(lib, package, package.lib):
 shared object ‘jsonlite.so’ not found
In addition: Warning message:
S3 methods ‘print.json’, ‘print.scalar’ were declared in NAMESPACE but not found

Do you have anymore suggestions on what I can do? Or is this a bug? I successfully installed GraalVM+FastR 20.3.0 and this package on an Ubuntu 20.04 docker container, so I am quite puzzled right now why it fails in macOS.

steve-s commented 3 years ago

Hello,

this is certainly weird. It looks like as if jsonlite.so was not produced at all, the C sources compilation step is completely missing in the log, it should look something like this:

* installing *source* package ‘jsonlite’ ...
** package ‘jsonlite’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
"/home/steve/Software/graalvm-ce-java8-21.0.0/jre/languages/llvm/native/bin/graalvm-native-clang" -I"/home/steve/Software/graalvm-ce-java8-21.0.0/jre/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fpic  -O2  -DFASTR -DNO_GNUR  -c base64.c -o base64.o
"/home/steve/Software/graalvm-ce-java8-21.0.0/jre/languages/llvm/native/bin/graalvm-native-clang" -I"/home/steve/Software/graalvm-ce-java8-21.0.0/jre/languages/R/include" -DNDEBUG -Iyajl/api  -O2 -DFASTR -DNO_GNUR  -fvisibility=hidden  -fpic  -O2  -DFASTR -DNO_GNUR  -c collapse_array.c -o collapse_array.o
.....

I have no idea why R would skip this step (package installation is implemented in base R packages that we take from GNU-R mostly unmodified). Do you use the native or LLVM backend? Could it be that you have somehow corrupted GRAALVM_HOME/languages/R/etc/Makeconf?

hsselman commented 3 years ago

I left out the lines between ** libs and ** R. Sorry for not mentioning it! Most certainly is this step executed. I left it out because it contained no obvious errors or warnings, and it does include my own path. I added ... in original post to stipulate it.

If I would need to add it, I would need to anonimize it. If it would help, I would be more than happy to do so. Let me know.

hsselman commented 3 years ago

I use native backend by setting it: fastr.setToolchain('native')

steve-s commented 3 years ago

If I would need to add it, I would need to anonimize it. If it would help, I would be more than happy to do so. Let me know.

yes, sorry that it means extra work, but I think it may shed some light on this. I suspect that the other issues that you just opened may have the same root cause.

hsselman commented 3 years ago

@steve-s I added the lines in between in all three issues!

steve-s commented 3 years ago

Hello,

so finally we can reproduce the issue and we'll look into it.

hsselman commented 3 years ago

Thanks @steve-s !

steve-s commented 3 years ago

Hello,

@fangerer managed to reproduce the issue and it seems the immediate cause is the -fvisibility=hidden. It is not clear to us, why this causes the problem and we'll need to analyze this bit more to do proper fix, but in the meantime you can edit $GRAALVM_HOME/languages/R/etc/Makeconf and remove the "-fvisibility=hidden" string from it.