search

Rdatatable / data.table

R's data.table package extends data.frame:
http://r-datatable.com
Mozilla Public License 2.0
3.63k stars 987 forks source link

data.table installation from source fails on Mac, `symbol not found in flat namespace '___kmpc_barrier'` #6622

Open dvg-p4 opened 1 week ago

dvg-p4 commented 1 week ago

This is likely a result of the changes introduced to solve https://github.com/Rdatatable/data.table/issues/6409.

If there's anything I need to change on my end to fix this, please let me know!

Minimal reproducible example

dgealow@Dans-P4-MacBook-Pro ~ % R-4.2-arm64 
dgealow@Dans-P4-MacBook-Pro ~ % R          

R version 4.2.3 (2023-03-15) -- "Shortstop Beagle"
[...]

> install.packages('data.table')
Installing package into ‘/Users/dgealow/Library/R/arm64/4.2/library’
(as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session ---
[selected 0-cloud]

  There is a binary version available but the source version is later:
           binary source needs_compilation
data.table 1.15.4 1.16.2              TRUE

Do you want to install from sources the package which needs compilation? (Yes/no/cancel) yes
installing the source package ‘data.table’
Full output

``` trying URL 'https://cloud.r-project.org/src/contrib/data.table_1.16.2.tar.gz' Content type 'application/x-gzip' length 5490076 bytes (5.2 MB) ================================================== downloaded 5.2 MB * installing *source* package ‘data.table’ ... ** package ‘data.table’ successfully unpacked and MD5 sums checked ** using staged installation zlib 1.2.11 is available ok * checking if R installation supports OpenMP with "-Xclang -fopenmp" ... no * checking if R installation supports OpenMP with "-fopenmp" ... yes ** libs clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c assign.c -o assign.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c between.c -o between.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c bmerge.c -o bmerge.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c chmatch.c -o chmatch.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c cj.c -o cj.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c coalesce.c -o coalesce.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c dogroups.c -o dogroups.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c fastmean.c -o fastmean.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c fcast.c -o fcast.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c fifelse.c -o fifelse.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c fmelt.c -o fmelt.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c forder.c -o forder.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c frank.c -o frank.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c fread.c -o fread.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c freadR.c -o freadR.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c froll.c -o froll.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c frollR.c -o frollR.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c frolladaptive.c -o frolladaptive.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c fsort.c -o fsort.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c fwrite.c -o fwrite.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c fwriteR.c -o fwriteR.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c gsumm.c -o gsumm.o gsumm.c:1173:38: warning: implicit conversion from 'long long' to 'long double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion] 1173 | ansd[i] = (ISNAN(s[i]) || s[i]>INT64_MAX || s[i]<=INT64_MIN) ? NA_INTEGER64 : (int64_t)s[i]; | ~^~~~~~~~~ /Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/stdint.h:94:26: note: expanded from macro 'INT64_MAX' 94 | #define INT64_MAX 9223372036854775807LL | ^~~~~~~~~~~~~~~~~~~~~ 1 warning generated. clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c idatetime.c -o idatetime.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c ijoin.c -o ijoin.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c init.c -o init.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c inrange.c -o inrange.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c nafill.c -o nafill.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c negate.c -o negate.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c nqrecreateindices.c -o nqrecreateindices.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c openmp-utils.c -o openmp-utils.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c programming.c -o programming.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c quickselect.c -o quickselect.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c rbindlist.c -o rbindlist.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c reorder.c -o reorder.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c shift.c -o shift.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c snprintf.c -o snprintf.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c subset.c -o subset.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c transpose.c -o transpose.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c types.c -o types.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c uniqlist.c -o uniqlist.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c utils.c -o utils.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c vecseq.c -o vecseq.o clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/R/arm64/include -fopenmp -fPIC -falign-functions=64 -Wall -g -O2 -c wrappers.c -o wrappers.o clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation ld: warning: -single_module is obsolete ld: warning: -multiply_defined is obsolete ld: warning: -single_module is obsolete ld: warning: -multiply_defined is obsolete PKG_CFLAGS = -fopenmp PKG_LIBS = -lz if [ "data.table.so" != "data_table.so" ]; then mv data.table.so data_table.so; fi if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id data_table.so data_table.so; fi installing to /Users/dgealow/Library/R/arm64/4.2/library/00LOCK-data.table/00new/data.table/libs ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices ** building package indices ** installing vignettes ** testing if installed package can be loaded from temporary location Error: package or namespace load failed for ‘data.table’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/Users/dgealow/Library/R/arm64/4.2/library/00LOCK-data.table/00new/data.table/libs/data_table.so': dlopen(/Users/dgealow/Library/R/arm64/4.2/library/00LOCK-data.table/00new/data.table/libs/data_table.so, 0x0006): symbol not found in flat namespace '___kmpc_barrier' Error: loading failed Execution halted ERROR: loading failed * removing ‘/Users/dgealow/Library/R/arm64/4.2/library/data.table’ * restoring previous ‘/Users/dgealow/Library/R/arm64/4.2/library/data.table’ The downloaded source packages are in ‘/private/var/folders/sr/lw2ng95s3mj1qsqtdxywzjvc0000gn/T/Rtmp0Drt3Z/downloaded_packages’ Warning message: In install.packages("data.table") : installation of package ‘data.table’ had non-zero exit status > ```

Output of sessionInfo()

R version 4.2.3 (2023-03-15)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 15.1.1

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.2.3
aitap commented 16 hours ago

Does it work if you Sys.setenv(PKG_LIBS='-fopenmp') first? If yes, we might need an export PKG_LIBS="${PKG_LIBS} -fopenmp" here too: https://github.com/Rdatatable/data.table/blob/546259ddaba0e8ab1506729113688f85ca2986fd/configure#L128

(Interesting how it previously worked when compiling with -fopenmp but linking without it. This is probably better handled by someone with a Mac.)