search

TheCacophonyProject / classifier-pipeline

Exports tracked animals through thermal vision.
GNU General Public License v3.0
21 stars 14 forks source link

Use h5py compression #59

Closed mjs closed 5 years ago

mjs commented 5 years ago

Use h5py's compression features. As well as saving disk space it may also result in net I/O performance wins.

http://docs.h5py.org/en/stable/high/dataset.html?highlight=compression

As part this, get rid of the remaining pytables dependencies.

gferraro commented 5 years ago

On a simple test found that writing to h5py file, is roughly twice as slow with compression.