Open pat-s opened 4 years ago
I solved the problem installing fst via with_makevars
to set the path to llvm:
v<-c("-L/usr/local/opt/llvm/lib","-I/usr/local/opt/llvm/include")
names(v)=c("LDFLAGS","CPPFLAGS")
with_makevars(v, install.packages("fst"), path = file.path("~", ".R", "Makevars"), c("="))```
@gabrimaine I confirm that this works, thanks! Maybe this can help @MarcusKlik to implement a generic fix.
Hi @gabrimaine, thanks for sharing your solution and @pat-s thanks for testing!
I've activated the R-CMD-check action
on the fstcore
repository to reproduce your error, see here. We can use that to test a fix. The thing is, I can't just add the LDFLAGS
and CPPFLAGS
to the package Makevars
, as users might not have installed the llvm
compiler or use another install location...
Looks like an OpenMP issue to me, i.e. the wrong pointers are set / or the wrong are found during source installation.
If I install the binary, OpenMP is also not detected (which is expected given that {data.table} also does not find it by default).
library(fst)
fst package v0.9.4
(OpenMP was not detected, using single threaded mode)
Not sure how {fst} does it but maybe looking how other popular packages (e.g. data.table) handle OpenMP might be fruitful?
See also https://mac.r-project.org/openmp/.
Following these instructions I can install {data.table} with OpenMP support.
In my case, I have OpenMp installed via brew install libomp
.
The important pieces are
CPPFLAGS= -I/usr/local/opt/libomp/include -Xclang -fopenmp
LDFLAGS += -lomp
Note that the first piece -I/usr/local/opt/libomp/include
is specific to detect the local libomp
install.
library(data.table)
data.table 1.13.0 using 4 threads (see ?getDTthreads). Latest news: r-datatable.com
# fst still errors
@pat-s I also encountered this issue and I solved it with adding CXX11 = $(LLVM_LOC)/bin/g++ -fopenmp
in Makevars
file. For details, please refer to https://github.com/Rdatatable/data.table/wiki/Installation
Thanks.
For many reasons this should be supported OOB in the package using Apples default clang compiler. The issue is still present with the new 11.0 SDK.
The reason that CRAN checks are not erroring probably relies on CRAN checking on macOS 10.13 and not against the most recent SDKs.
Hi @pat-s, you're definitely right, this should work OOB for macOS users. The data.table
package is using a configure script now to detect OpenMP on macOS and I think it's a good idea to use a similar strategy for fst
to avoid these installation problems.
Package fstcore
is now tested on macOS with and without the bundled fstlib
libraries. Please let me know if there are any more problems compiling the package on MacOS.
Should be the same as #241 but with a more descriptive title.
I am facing this issue locally since months - in addition, the GitHub Actions install failures cause headaches from time to time. Time to put some weight onto it :)
I saw you moved the issue to the next milestone. Do you have any idea when it will be tackled?
Note that CRAN tests on macOS 10.13 and the latest SDKs are somewhat troublesome for source installs. Hence, while your package might install properly on CRAN, it will error on 90% of all user machines when doing source installs because they are > 10.13.
It does not need to be SDK releated necessarily. Maybe some brew linkings gets into the way. However, given that the minimal GitHub Actions builds also show the error, I am somewhat confident that it is not related to a local config problem of mine :)
The error below uses the default clang from macOS (on macOS 10.15), as it is the standard for R >= 4.0.