apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.56k stars 3.54k forks source link

[R] write_parquet() doesn't work in R package 15.0.1 update #40726

Closed aaronschiff closed 7 months ago

aaronschiff commented 7 months ago

Describe the bug, including details regarding any error messages, version, and platform.

On MacOS, after I updated the arrow R package to 15.0.1, write_parquet() fails:

r$> write_parquet(x = data.frame(a = c(1, 2, 3)), sink = "test.parquet")
Error in parquet___WriterProperties___Builder__create() : 
  Cannot call parquet___WriterProperties___Builder__create(). See https://arrow.apache.org/docs/r/articles/install.html for help installing Arrow C++ libraries. 

It was working prior to the update (version 15.0.0, I think). I tried removing and re-installing the package but it didn't help. The read_parquet() function works fine however.

Output from arrow_info():

r$> arrow_info()
Arrow package version: 15.0.1

Capabilities:

acero      TRUE
dataset   FALSE
substrait FALSE
parquet   FALSE
json      FALSE
s3        FALSE
gcs       FALSE
utf8proc  FALSE
re2       FALSE
snappy    FALSE
gzip      FALSE
brotli    FALSE
zstd      FALSE
lz4       FALSE
lz4_frame FALSE
lzo       FALSE
bz2       FALSE
jemalloc  FALSE
mimalloc  FALSE

Memory:

Allocator  system
Current   0 bytes
Max       0 bytes

Runtime:

SIMD Level          none
Detected SIMD Level none

Build:

C++ Library Version           15.0.1
C++ Compiler              AppleClang
C++ Compiler Version 14.0.0.14000029

Component(s)

R

aaronschiff commented 7 months ago

Sorry, I just saw this has already been reported as part of a separate issue

stevecondylios commented 7 months ago

First time user of arrow, getting the same error. Is the problem with arrow itself or with the arrow R package? (trying to work out if I should install a previous version of arrow itself, or just the arrow R package). Are you able to link to the issue where it has already been reported? (I couldn't spot it).

aaronschiff commented 7 months ago

@stevecondylios see here for a fix: https://github.com/apache/arrow/issues/40667

It's an issue with the version 15.0.1 package on CRAN

stevecondylios commented 7 months ago

@aaronschiff thanks a lot, got it working 2 minutes after reading your reply.

In case anyone else ends up here, you temporarily need to install like so:

install.packages('arrow', repos = c('https://apache.r-universe.dev'))

(source)