search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.56k stars 3.54k forks source link

[R] arrow R package 15.0.1 missing dataset, parquet, etc. on Mac M2 #40667

Closed blongworth closed 7 months ago

blongworth commented 7 months ago

Describe the bug, including details regarding any error messages, version, and platform.

After installing Arrow in R, the Arrow binary looks like it was compiled without many features.

Arrow 14.0.2 was working before updating.

I had previously installed arrow and R via homebrew, but had switched to installing R via brew install --cask r and arrow via install.packages("arrow") from within R. Homebrew arrow was still installed until yesterday, but I think R was using the binary installed with the arrow package. Giving this background in case this is specific to something about my setup.

System: Apple M2 Pro, Sonoma 14.4

R info:

R version 4.3.3 (2024-02-29) -- "Angel Food Cake"
Platform: aarch64-apple-darwin20 (64-bit)

arrow::arrow_info() output:

Arrow package version: 15.0.1

Capabilities:

acero      TRUE
dataset   FALSE
substrait FALSE
parquet   FALSE
json      FALSE
s3        FALSE
gcs       FALSE
utf8proc  FALSE
re2       FALSE
snappy    FALSE
gzip      FALSE
brotli    FALSE
zstd      FALSE
lz4       FALSE
lz4_frame FALSE
lzo       FALSE
bz2       FALSE
jemalloc  FALSE
mimalloc  FALSE

Memory:

Allocator  system
Current   0 bytes
Max       0 bytes

Runtime:

SIMD Level          none
Detected SIMD Level none

Build:

C++ Library Version           15.0.1
C++ Compiler              AppleClang
C++ Compiler Version 14.0.0.14000029

Component(s)

R

letitbk commented 7 months ago

I have the same issue here. I tried many things including removing and reinstalling R from scratch again including install arrow package using install_arrow() with many different options, and all failed.

blongworth commented 7 months ago

I was able to install arrow 15.0.1 on a linux docker container (rocker/rstudio). The compiled arrow had parquet and dataset enabled. Is there an easy way to tell R arrow to use a different binary? I'd like to test the homebrew version installed by brew install apache-arrow.

blongworth commented 7 months ago

I am able to get a working install of arrow 15.0.2.100000328 by using:arrow::install_arrow(nightly = TRUE)

grantmcdermott commented 7 months ago

Same issue here. Tried both the Mac (Intel chip) binary builds from CRAN and R-universe.

letitbk commented 7 months ago

I am able to get a working install of arrow 15.0.2.100000328 by using:arrow::install_arrow(nightly = TRUE)

I couldn't, and here's the error message

Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-arrow/00new/arrow/libs/arrow.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-arrow/00new/arrow/libs/arrow.so, 0x0006): symbol not found in flat namespace '__ZN5arrow12ArrayBuilder13AppendScalarsERKSt6vectorISt10shared_ptrINS_6ScalarEESaIS4_EE'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/arrow’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/arrow’
blongworth commented 7 months ago

@letitbk, given the 00LOCK-arrow path, I wonder if a library was in use? I ran install_arrow() from R running in a shell, no other instances running. It's been a while since I've dealt with R 00LOCK stuff, so I could be wrong.

letitbk commented 7 months ago

@blongworth thanks for your suggestion, but I encountered the another message, and here's the entire error message. 

> arrow::install_arrow(nightly = TRUE)
trying URL 'https://nightlies.apache.org/arrow/r/src/contrib/arrow_15.0.2.100000328.tar.gz'
Content type 'application/x-gzip' length 4238620 bytes (4.0 MB)
==================================================
downloaded 4.0 MB

* installing *source* package ‘arrow’ ...
** using staged installation
*** Disabling Arrow build with GCS on gcc-13.
*** Set ARROW_GCS=ON to explicitly enable.
*** pkg-config found.
*** Latest available nightly for 15.0.2.100000328: 15.0.2.100000328
*** Found libcurl and OpenSSL >= 3.0.0
*** Successfully retrieved libarrow (darwin-arm64-openssl-3.0)
PKG_CFLAGS=-DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS
PKG_LIBS=-L/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/lib -L/opt/homebrew/opt/openssl@3/lib -larrow_dataset -lparquet -larrow_acero -larrow -larrow_bundled_dependencies -framework Security -lcurl -lssl -lcrypto 
** libs
using C++ compiler: ‘Homebrew clang version 17.0.6’
using C++17
using SDK: ‘MacOSX14.4.sdk’
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c RTasks.cpp -o RTasks.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c altrep.cpp -o altrep.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c array.cpp -o array.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c array_to_vector.cpp -o array_to_vector.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c arraydata.cpp -o arraydata.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c arrowExports.cpp -o arrowExports.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c bridge.cpp -o bridge.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c buffer.cpp -o buffer.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c chunkedarray.cpp -o chunkedarray.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c compression.cpp -o compression.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c compute-exec.cpp -o compute-exec.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c compute.cpp -o compute.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c config.cpp -o config.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c csv.cpp -o csv.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c dataset.cpp -o dataset.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c datatype.cpp -o datatype.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c expression.cpp -o expression.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c extension-impl.cpp -o extension-impl.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c feather.cpp -o feather.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c field.cpp -o field.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c filesystem.cpp -o filesystem.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c io.cpp -o io.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c json.cpp -o json.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c memorypool.cpp -o memorypool.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c message.cpp -o message.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c parquet.cpp -o parquet.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c r_to_arrow.cpp -o r_to_arrow.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c recordbatch.cpp -o recordbatch.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c recordbatchreader.cpp -o recordbatchreader.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c recordbatchwriter.cpp -o recordbatchwriter.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c safe-call-into-r-impl.cpp -o safe-call-into-r-impl.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c scalar.cpp -o scalar.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c schema.cpp -o schema.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c symbols.cpp -o symbols.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c table.cpp -o table.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c threadpool.cpp -o threadpool.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -DPARQUET_STATIC -DARROW_STATIC -I/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/include -I/opt/homebrew/opt/openssl@3/include   -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_ACERO -DARROW_R_WITH_JSON -DARROW_R_WITH_S3 -DARROW_R_WITH_GCS -I'/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/cpp11/include' -I/opt/homebrew/opt/gettext/include -I/opt/homebrew/opt/llvm/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c type_infer.cpp -o type_infer.o
/opt/homebrew/bin/g++-13 -fopenmp -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/homebrew/opt/gettext/lib -L/opt/homebrew/opt/llvm/lib -Wl,-rpath,/opt/homebrew/opt/llvm/lib -o arrow.so RTasks.o altrep.o array.o array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o expression.o extension-impl.o feather.o field.o filesystem.o io.o json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o schema.o symbols.o table.o threadpool.o type_infer.o -L/private/var/folders/7c/9klc1txn7ml64wkb2zcq24_w0000gr/T/RtmpFAtig8/R.INSTALL60b457156403/arrow/libarrow/arrow-15.0.2.100000328/lib -L/opt/homebrew/opt/openssl@3/lib -larrow_dataset -lparquet -larrow_acero -larrow -larrow_bundled_dependencies -framework Security -lcurl -lssl -lcrypto -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: search path '/opt/homebrew/opt/gettext/lib' not found
ld: warning: reexported library with install name '/opt/homebrew/opt/llvm/lib/libunwind.1.dylib' found at '/opt/homebrew/Cellar/llvm/17.0.6_1/lib/libunwind.1.0.dylib' couldn't be matched with any parent library and will be linked directly
installing to /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-arrow/00new/arrow/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-arrow/00new/arrow/libs/arrow.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-arrow/00new/arrow/libs/arrow.so, 0x0006): symbol not found in flat namespace '__ZN5arrow12ArrayBuilder13AppendScalarsERKSt6vectorISt10shared_ptrINS_6ScalarEESaIS4_EE'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/arrow’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/arrow’

Perhaps the issue is related to the version mismatch?

amoeba commented 7 months ago

Hi all, sorry for the trouble. The most recent submission (15.0.1) ran into some trouble after acceptance when being built on macOS. This lead to the reduced feature-set. We're actively working on preparing a new submission so the macOS version of the package has the usual feature-set but it may take some time.

In the mean time, 15.0.1 is now on R-universe so you should be able to install a binary using:

install.packages('arrow', repos = c('https://apache.r-universe.dev'))

@letitbk I'm not sure what's going on with your issue directly above but can you try installing from R-universe and report back?

blongworth commented 7 months ago

The new version from r-universe installs fine and is compiled with all needed features. Thank you!!

@letitbk, agree with @amoeba about trying the r-universe version, but it looks like your compile can't find some needed libraries it's expecting to find in homebrew's install path. Getting a working precompiled arrow binary is much easier.

I had issues with R installed via brew install r and arrow a while back. Make sure your R is installed from CRAN or via brew install --cask r to avoid some odd binary incompatibility issues.

letitbk commented 7 months ago

thank you so much @blongworth the new version from r-universe installs fine for me as well!!

blongworth commented 7 months ago

Thank @amoeba and the Arrow folks. A couple of hours turn around for new builds is pretty great! CRAN, of course, is another story.

amoeba commented 7 months ago

I'm going to close this since https://github.com/apache/arrow/pull/41019 was just merged which adds a warning with instructions for how to resolve situations like this in the future (by installing from r-universe). That change will be included in Arrow 16 which should be out in the earlier part of Q2. I think the hope is still that the ultimate issue with CRAN macOS binaries can be addressed but future versions of the package will at least help you out if/when this happens again.

Thanks all for the reports.