Need an API to be able to control the PyTables file object, in case you don't want to save to disk

uchicago-cs / deepdish

Flexible HDF5 saving/loading and other data science tools from the University of Chicago

http://deepdish.io

BSD 3-Clause "New" or "Revised" License

270 stars 59 forks source link

Need an API to be able to control the PyTables file object, in case you don't want to save to disk #30

Open aijoe-mcarthyfinch opened 6 years ago

aijoe-mcarthyfinch commented 6 years ago

I have a use-case where I don't want to save the resulting HDF5 data to local disk. Rather, I want to handle the produced data with a stream pattern.

In my specific case, this is so I can save the HDF5 data directly to an AWS S3 bucket in a resource-efficient way, but I imaging there are other situations where people may want to have control of the file handle. It would be good to have an extended API which supports this.