delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python
https://delta-io.github.io/delta-rs/
Apache License 2.0
2.33k stars 411 forks source link

Provide documentation how to configure various storage backends #2748

Open roeap opened 3 months ago

roeap commented 3 months ago

Description

While the "build-in" backends adopt the configuration of object_store and we could always refer to the config over there, we should add some docs in the delta-rs documentation at least pointing folks in the right direction.

Specifically the S3 integration can be adopted to work with several other implementations, sometimes leveraging features (atomic rename) not available in "vanilla" S3.

Use Case

Users should easily discover how they can leverage their storage backend.

Related Issue(s)

2361