rajasekarv / vega

A new arguably faster implementation of Apache Spark from scratch in Rust
Apache License 2.0
2.23k stars 206 forks source link

Add connector for AWS S3 #51

Open iduartgomez opened 4 years ago

iduartgomez commented 4 years ago

Implement a connector to read/write from/to AWS S3. For inspiration maybe look at a the HDFS FS interface. If possible try to come with a common interface we could reuse for other cloud providers (and potentially any "fs-like" source).