search

datafusion-contrib / bdt

Boring Data Tool
Apache License 2.0
210 stars 20 forks source link

Add cloud object store support (S3 etc) #10

Open andygrove opened 1 year ago

YuriyGavrilov commented 1 year ago

Maybe add support of storj network through uplink https://github.com/storj/uplink

trungda commented 9 months ago

Seems this is still a nice-to-have feature? I'd like to take a stab at this to understand more the datafusion ecosystem.

andygrove commented 9 months ago

Seems this is still a nice-to-have feature? I'd like to take a stab at this to understand more the datafusion ecosystem.

Yes, it is. DataFusion supports reading from S3 and other object stores already, so hopefully this is not too hard to add.

ForeverAngry commented 6 months ago

@trungda I'm interested in this feature. Is this something you have solved?

trungda commented 6 months ago

@trungda I'm interested in this feature. Is this something you have solved?

No I haven't yet. It's on my list but I don't have time to get to it in the next month. Feel free to jump into it?