search

datafusion-contrib / bdt

Boring Data Tool
Apache License 2.0
198 stars 18 forks source link

Add cloud object store support (S3 etc) #10

Open andygrove opened 1 year ago

YuriyGavrilov commented 8 months ago

Maybe add support of storj network through uplink https://github.com/storj/uplink

trungda commented 4 months ago

Seems this is still a nice-to-have feature? I'd like to take a stab at this to understand more the datafusion ecosystem.

andygrove commented 4 months ago

Seems this is still a nice-to-have feature? I'd like to take a stab at this to understand more the datafusion ecosystem.

Yes, it is. DataFusion supports reading from S3 and other object stores already, so hopefully this is not too hard to add.

ForeverAngry commented 1 month ago

@trungda I'm interested in this feature. Is this something you have solved?

trungda commented 1 month ago

@trungda I'm interested in this feature. Is this something you have solved?

No I haven't yet. It's on my list but I don't have time to get to it in the next month. Feel free to jump into it?