search

kbajalc / parquets

MIT License
44 stars 21 forks source link

Load directly from S3 #7

Open dobesv opened 4 years ago

dobesv commented 4 years ago

It would be nice to port over the S3 support from https://github.com/ZJONSSON/parquetjs

Would you accept a PR for this?

moltar commented 3 years ago

Seems like unrelated concerns. I don't feel like a transform library should know about storage mechanisms.

dobesv commented 3 years ago

If it isn't aware of the storage mechanism, it might need some kind of plugin system for random access reads instead.

berenddeboer commented 2 years ago

I think what it needs is simply support for Readable, just like it has for Writable?