search

fstpackage / fst

Lightning Fast Serialization of Data Frames for R
http://www.fstpackage.org/fst/
GNU Affero General Public License v3.0
614 stars 42 forks source link

OpenMP not detected Mac 12 (Monterey) M1 (ARM) Mac #263

Open markdanese opened 2 years ago

markdanese commented 2 years ago

I installed fst on a clean build of a new Mac M1 computer running OS 12 (Monterey). Version 0.9.4. I originally had trouble with data.table but finally got the Makevars working and data.table loads and recognizes OpenMP (see this link). But I can't seem to get fst to recognize OpenMP. This is way outside my area of expertise so I don't have any solutions to offer but wanted to at least put this out there for others. It may be related to the how homebrew is now installing libraries or the new Mac OS or to the ARM processor (or Rosetta).

I should clarify that I installed fst from source and it seems to have compiled correctly. And it works fine, just not with OpenMP.

> library(data.table)
data.table 1.14.2 using 5 threads (see ?getDTthreads).  Latest news: r-datatable.com
> library(fst)  
fst package v0.9.4
(OpenMP was not detected, using single threaded mode)
dxf1david commented 2 years ago

Yes - I'm having the same problem with fst on a 4-core intel iMac - fst installs without errors from source, but it doesn't recognize openMP

sohrabsa commented 2 years ago

similar to @markdanese, configuring ~/.R/Makevars works for getting data.table to recognize openMP but not for fst. I have also tried instructions #242, see comments by mimaque to no avail. Would appreciate any help. MacOS Intel, BigSur version 11.6.1 and R version 4.1.0 (2021-05-18) -- "Camp Pontanezen".

sohrabsa commented 2 years ago

Just adding a bit more detail. Here's the result after running install.packages('fst', type = 'source'):

trying URL 'https://mirror.las.iastate.edu/CRAN/src/contrib/fst_0.9.4.tar.gz'
Content type 'application/x-gzip' length 564903 bytes (551 KB)
==================================================
downloaded 551 KB

* installing *source* package ‘fst’ ...
** package ‘fst’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c RcppExports.cpp -o RcppExports.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c flex_store.cpp -o flex_store.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fst_blockrunner_char.cpp -o fst_blockrunner_char.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fst_compress.cpp -o fst_compress.o
In file included from fst_compress.cpp:30:
fstcore/interface/fstcompressor.h:167: warning: ignoring ‘#pragma omp parallel’ [-Wunknown-pragmas]
  167 | #pragma omp parallel num_threads(nr_of_threads) shared(comp_sizes,nr_of_threads,comp_batch_sizes)
      | 
fstcore/interface/fstcompressor.h:169: warning: ignoring ‘#pragma omp for’ [-Wunknown-pragmas]
  169 | #pragma omp for schedule(static, 1) nowait
      | 
fstcore/interface/fstcompressor.h:204: warning: ignoring ‘#pragma omp single’ [-Wunknown-pragmas]
  204 | #pragma omp single
      | 
fstcore/interface/fstcompressor.h:305: warning: ignoring ‘#pragma omp parallel’ [-Wunknown-pragmas]
  305 | #pragma omp parallel for schedule(static, 1)
      | 
fstcore/interface/fstcompressor.h:414: warning: ignoring ‘#pragma omp parallel’ [-Wunknown-pragmas]
  414 | #pragma omp parallel num_threads(nrOfThreads)
      | 
fstcore/interface/fstcompressor.h:416: warning: ignoring ‘#pragma omp for’ [-Wunknown-pragmas]
  416 | #pragma omp for schedule(static, 1) nowait
      | 
fstcore/interface/fstcompressor.h:432: warning: ignoring ‘#pragma omp single’ [-Wunknown-pragmas]
  432 | #pragma omp single
      | 
fstcore/interface/fstcompressor.h:463: warning: ignoring ‘#pragma omp parallel’ [-Wunknown-pragmas]
  463 | #pragma omp parallel num_threads(nrOfThreads)
      | 
fstcore/interface/fstcompressor.h:465: warning: ignoring ‘#pragma omp for’ [-Wunknown-pragmas]
  465 | #pragma omp for schedule(static, 1) nowait
      | 
fstcore/interface/fstcompressor.h:487: warning: ignoring ‘#pragma omp single’ [-Wunknown-pragmas]
  487 | #pragma omp single
      | 
In file included from fst_compress.cpp:31:
fstcore/interface/fsthash.h:100: warning: ignoring ‘#pragma omp parallel’ [-Wunknown-pragmas]
  100 | #pragma omp parallel num_threads(nrOfThreads)
      | 
fstcore/interface/fsthash.h:102: warning: ignoring ‘#pragma omp for’ [-Wunknown-pragmas]
  102 | #pragma omp for schedule(static, 1) nowait
      | 
fstcore/interface/fsthash.h:117: warning: ignoring ‘#pragma omp single’ [-Wunknown-pragmas]
  117 | #pragma omp single
      | 
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fst_error.cpp -o fst_error.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fst_table.cpp -o fst_table.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c init.c -o init.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c openmp.cpp -o openmp.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/LZ4/lz4.c -o fstcore/LZ4/lz4.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/LZ4/xxhash.c -o fstcore/LZ4/xxhash.o
ar rcs libLZ4.a fstcore/LZ4/lz4.o fstcore/LZ4/xxhash.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/common/entropy_common.c -o fstcore/ZSTD/common/entropy_common.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/common/error_private.c -o fstcore/ZSTD/common/error_private.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/common/fse_decompress.c -o fstcore/ZSTD/common/fse_decompress.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/fse_compress.c -o fstcore/ZSTD/compress/fse_compress.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/zstd_compress_literals.c -o fstcore/ZSTD/compress/zstd_compress_literals.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/zstd_compress_sequences.c -o fstcore/ZSTD/compress/zstd_compress_sequences.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/decompress/huf_decompress.c -o fstcore/ZSTD/decompress/huf_decompress.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/huf_compress.c -o fstcore/ZSTD/compress/huf_compress.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/decompress/zstd_ddict.c -o fstcore/ZSTD/decompress/zstd_ddict.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/decompress/zstd_decompress.c -o fstcore/ZSTD/decompress/zstd_decompress.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/decompress/zstd_decompress_block.c -o fstcore/ZSTD/decompress/zstd_decompress_block.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/common/zstd_common.c -o fstcore/ZSTD/common/zstd_common.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/common/debug.c -o fstcore/ZSTD/common/debug.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/zstd_compress.c -o fstcore/ZSTD/compress/zstd_compress.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/dictBuilder/cover.c -o fstcore/ZSTD/dictBuilder/cover.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/dictBuilder/divsufsort.c -o fstcore/ZSTD/dictBuilder/divsufsort.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/zstd_fast.c -o fstcore/ZSTD/compress/zstd_fast.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/zstd_lazy.c -o fstcore/ZSTD/compress/zstd_lazy.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/zstd_ldm.c -o fstcore/ZSTD/compress/zstd_ldm.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/common/pool.c -o fstcore/ZSTD/common/pool.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/zstd_opt.c -o fstcore/ZSTD/compress/zstd_opt.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/dictBuilder/zdict.c -o fstcore/ZSTD/dictBuilder/zdict.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/zstd_double_fast.c -o fstcore/ZSTD/compress/zstd_double_fast.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/compress/hist.c -o fstcore/ZSTD/compress/hist.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c fstcore/ZSTD/dictBuilder/fastcover.c -o fstcore/ZSTD/dictBuilder/fastcover.o
ar rcs libZSTD.a fstcore/ZSTD/common/entropy_common.o fstcore/ZSTD/common/error_private.o fstcore/ZSTD/common/fse_decompress.o fstcore/ZSTD/compress/fse_compress.o fstcore/ZSTD/compress/zstd_compress_literals.o fstcore/ZSTD/compress/zstd_compress_sequences.o fstcore/ZSTD/decompress/huf_decompress.o fstcore/ZSTD/compress/huf_compress.o fstcore/ZSTD/decompress/zstd_ddict.o fstcore/ZSTD/decompress/zstd_decompress.o fstcore/ZSTD/decompress/zstd_decompress_block.o fstcore/ZSTD/decompress/zstd_decompress.o fstcore/ZSTD/common/zstd_common.o fstcore/ZSTD/common/debug.o fstcore/ZSTD/compress/zstd_compress.o fstcore/ZSTD/dictBuilder/cover.o fstcore/ZSTD/dictBuilder/divsufsort.o fstcore/ZSTD/compress/zstd_fast.o fstcore/ZSTD/compress/zstd_lazy.o fstcore/ZSTD/compress/zstd_ldm.o fstcore/ZSTD/common/pool.o fstcore/ZSTD/compress/zstd_opt.o fstcore/ZSTD/dictBuilder/zdict.o fstcore/ZSTD/compress/zstd_double_fast.o fstcore/ZSTD/compress/hist.o fstcore/ZSTD/dictBuilder/fastcover.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/compression/compression.cpp -o fstcore/compression/compression.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/compression/compressor.cpp -o fstcore/compression/compressor.o
fstcore/compression/compressor.cpp:318: warning: ignoring ‘#pragma omp critical’ [-Wunknown-pragmas]
  318 |         #pragma omp critical
      | 
fstcore/compression/compressor.cpp:344: warning: ignoring ‘#pragma omp critical’ [-Wunknown-pragmas]
  344 |         #pragma omp critical (criticalcompression)
      | 
fstcore/compression/compressor.cpp:366: warning: ignoring ‘#pragma omp critical’ [-Wunknown-pragmas]
  366 |         #pragma omp critical (criticalcompression)
      | 
ar rcs libCOMPRESSION.a fstcore/compression/compression.o fstcore/compression/compressor.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/interface/openmphelper.cpp -o fstcore/interface/openmphelper.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/interface/fststore.cpp -o fstcore/interface/fststore.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/logical/logical_v10.cpp -o fstcore/logical/logical_v10.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/integer/integer_v8.cpp -o fstcore/integer/integer_v8.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/byte/byte_v12.cpp -o fstcore/byte/byte_v12.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/double/double_v9.cpp -o fstcore/double/double_v9.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/character/character_v6.cpp -o fstcore/character/character_v6.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/factor/factor_v7.cpp -o fstcore/factor/factor_v7.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/blockstreamer/blockstreamer_v2.cpp -o fstcore/blockstreamer/blockstreamer_v2.o
fstcore/blockstreamer/blockstreamer_v2.cpp:114: warning: ignoring ‘#pragma omp parallel’ [-Wunknown-pragmas]
  114 | #pragma omp parallel num_threads(nrOfThreads)
      | 
fstcore/blockstreamer/blockstreamer_v2.cpp:116: warning: ignoring ‘#pragma omp for’ [-Wunknown-pragmas]
  116 | #pragma omp for ordered schedule(static, 1)
      | 
fstcore/blockstreamer/blockstreamer_v2.cpp:134: warning: ignoring ‘#pragma omp ordered’ [-Wunknown-pragmas]
  134 | #pragma omp ordered
      | 
fstcore/blockstreamer/blockstreamer_v2.cpp:284: warning: ignoring ‘#pragma omp parallel’ [-Wunknown-pragmas]
  284 | #pragma omp parallel num_threads(nrOfThreads)
      | 
fstcore/blockstreamer/blockstreamer_v2.cpp:286: warning: ignoring ‘#pragma omp for’ [-Wunknown-pragmas]
  286 | #pragma omp for ordered schedule(static, 1)
      | 
fstcore/blockstreamer/blockstreamer_v2.cpp:310: warning: ignoring ‘#pragma omp ordered’ [-Wunknown-pragmas]
  310 | #pragma omp ordered
      | 
fstcore/blockstreamer/blockstreamer_v2.cpp:736: warning: ignoring ‘#pragma omp parallel’ [-Wunknown-pragmas]
  736 | #pragma omp parallel num_threads(nrOfThreads) shared(isAlligned,nrOfBatches,batchSize,blockCount)
      | 
fstcore/blockstreamer/blockstreamer_v2.cpp:738: warning: ignoring ‘#pragma omp for’ [-Wunknown-pragmas]
  738 | #pragma omp for schedule(static, 1)
      | 
fstcore/blockstreamer/blockstreamer_v2.cpp:749: warning: ignoring ‘#pragma omp critical’ [-Wunknown-pragmas]
  749 | #pragma omp critical
      | 
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/integer64/integer64_v11.cpp -o fstcore/integer64/integer64_v11.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common -Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -fPIC  -Wall -g -O2  -c fstcore/byteblock/byteblock_v13.cpp -o fstcore/byteblock/byteblock_v13.o
ar rcs libFRAME.a fstcore/interface/openmphelper.o fstcore/interface/fststore.o fstcore/logical/logical_v10.o fstcore/integer/integer_v8.o fstcore/byte/byte_v12.o fstcore/double/double_v9.o fstcore/character/character_v6.o fstcore/factor/factor_v7.o fstcore/blockstreamer/blockstreamer_v2.o fstcore/integer64/integer64_v11.o fstcore/byteblock/byteblock_v13.o
/usr/local/gfortran/bin/g++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/opt/gettext/lib -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib -o fst.so RcppExports.o flex_store.o fst_blockrunner_char.o fst_compress.o fst_error.o fst_table.o init.o openmp.o -L. -lFRAME -lCOMPRESSION -lLZ4 -lZSTD -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/4.1/Resources/library/00LOCK-fst/00new/fst/libs
** R
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (fst)

The downloaded source packages are in
    ‘/private/var/folders/zv/4lhpb4hj0slb_51r6484tfy9zckf2s/T/Rtmp859iT0/downloaded_packages’
sohrabsa commented 2 years ago

And the output:

> library(fst)
fst package v0.9.4
(OpenMP was not detected, using single threaded mode)
sohrabsa commented 2 years ago

A quick hack to get it to work - ensure that g++ is called with -fopenmp, for my Makevars that means:

CXX1X=/usr/local/gfortran/bin/g++ -fopenmp
CXX98=/usr/local/gfortran/bin/g++ -fopenmp
CXX11=/usr/local/gfortran/bin/g++ -fopenmp
CXX14=/usr/local/gfortran/bin/g++ -fopenmp
CXX17=/usr/local/gfortran/bin/g++ -fopenmp

Now:

library(fst)
fst package v0.9.4
(OpenMP detected, using 12 threads)
markdanese commented 2 years ago

Unfortunately, I couldn't get that to work for me on an M1 Mac OSX 12.1. Doesn't mean I did it correctly, but I tried a number of variations replacing /usr/local with /opt/homebrew and couldn't get fst to recognize additional threads.

markdanese commented 2 years ago

My fault. It works fine if I build from source. Below are the extra lines I added to my existing Makevars

CXX1X=/opt/homebrew/bin/g++-11 -fopenmp
CXX98=/opt/homebrew/bin/g++-11 -fopenmp
CXX11=/opt/homebrew/bin/g++-11 -fopenmp
CXX14=/opt/homebrew/bin/g++-11 -fopenmp
CXX17=/opt/homebrew/bin/g++-11 -fopenmp

Then I get this:

> library(fst)
fst package v0.9.4
(OpenMP detected, using 10 threads)

And data.table is unaffected (still getting 5 threads).

MarcusKlik commented 2 years ago

Hi @markdanese, thanks for reporting your solution (and adding it to the data.table wiki)! Did fst also compile correctly when you use the clang compiler?

markdanese commented 2 years ago

I don't really understand compilers or the compilation process so I had somebody knowledgeable helping me figure it out. Hence, I can't answer your question. My first solution worked for fst, but fst didn't take advantage of multiple threads. When I added the above lines, that problem was fixed.

dxf1david commented 2 years ago

@markdanese - thanks very much for the info about your Makevars - I can finally use fst with multiple threads on my Mac

fox34 commented 2 years ago

Please note that since fst 0.9.8 the new package fstcore must (also) be installed with the abovementioned lines in Makevars:

install.packages("fstcore")
install.packages("fst")
markdanese commented 2 years ago

I can confirm that 0.9.8 is now back to single threaded. If someone can clarify what to change in Makevars, that would be the most helpful. I don't know what it means to "install fstcore with Makevars."

markdanese commented 2 years ago

Just to follow-up on my comment, one has to install fstcore from source: install.packages("fstcore", type = "source")

Now I get 10 threads.

fox34 commented 2 years ago

Just to follow-up on my comment, one has to install fstcore from source: install.packages("fstcore", type = "source")

N.B.: AFAIK, install from source should be the default for most installations. Most likely, your single-core-issue was resolved simply because of the re-installation and thus re-compilation of fstcore with your custom lines in Makevars.

Simply re-installing fst alone would not solve this issue, since an already installed single-core-fstcore would be unaffected (and not recompiled) in this case.

pat-s commented 2 years ago

https://github.com/fstpackage/fst/issues/242#issuecomment-1054634408 might help and might be more troublefree than using gcc.

mpr1255 commented 2 years ago

I just wanted to say that the instructions here solved it for me: https://investcookies.ru/post/datatable_m1/data_table_arm/ (I know, the website looks super sketchy lol). I reinstalled data.table and fst from source after making those suggested tweaks to Makevars (and brew install llvm) -- and they compiled and worked. Very cool!

dcaud commented 4 months ago

Just in case this can be fixed: this remains an issue for me.