search

apache / iceberg-go

Apache Iceberg - Go
https://iceberg.apache.org/
Apache License 2.0
143 stars 35 forks source link

IO Implementation using Go CDK #111

Open srilman opened 4 months ago

srilman commented 4 months ago

Partially implements #92. The Go CDK has well-maintained implementations for accessing objects stores from S3, Azure, and GCS via a io/fs.Fs-like interface. However, their file interface doesn't support the io.ReaderAt interface or the Seek() function that Iceberg-Go requires for files. Furthermore, the File components are private. So we copied the wrappers and implement the remaining functions inside of Iceberg-Go directly.

In addition, we add support for S3 Read IO using the CDK, providing the option to choose between the existing and new implementation using an extra property.

Next PR will implement write IO, at which point we will drop the non-CDK S3 IO implementation.

loicalleyne commented 2 months ago

Hi @srilman, is this still on your radar?