search

apache / arrow

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
https://arrow.apache.org/
Apache License 2.0
14.37k stars 3.49k forks source link

[C++] Enable using the GCS+GRPC plugin with Arrow #40254

Open anjakefala opened 7 months ago

anjakefala commented 7 months ago

Describe the enhancement requested

Link to plugin: https://cloud.google.com/cpp/docs/reference/storage/latest/storage-grpc

Getting the experimental gRPC+GCS plugin to work would require quite a bit build system (CMake) work and the upgrading of libprotobuf and gRPC dependencies.

This is untested and incomplete, but this is an effort that was made to get it working.

This could potentially help with higher throughput with GCSFS.

Component(s)

C++

benibus commented 5 months ago

This is tangential, but upgrading gRPC to v1.61.0 or later would also allow us to use the new OpenTelemetry plugin. I think we could handle the version bump(s) in a standalone PR first - assuming we wouldn't need to target a lower version.