search

ratal / mdfreader

Read Measurement Data Format (MDF) versions 3.x and 4.x file formats in python
Other
169 stars 74 forks source link

Added compression to HDF5 output #118

Closed cristi-neagu closed 6 years ago

cristi-neagu commented 6 years ago

Can now choose HDF5 dataset compression parameters. Level 4 gzip compression creates files about 8% the size while taking 2-3 times longer.

cristi-neagu commented 6 years ago

Depending on the performance drop, we could make gzip compression as default, instead of None. On my machine, with a 200MB mdf file, no compression was around 3s, while gzip level 4 was around 7 seconds. Considering that no compression created a 600MB file (3 times the size of the original mdf!) and with compression i got a 42MB file, i think the performance hit is definitely worth it.