search

AllenInstitute / bmtk

Brain Modeling Toolkit
https://alleninstitute.github.io/bmtk/
BSD 3-Clause "New" or "Revised" License
266 stars 86 forks source link

Adding compression capability in builder #300

Closed shixnya closed 1 year ago

shixnya commented 1 year ago

I added compression capability in the builder. The arguments are passed to create_dataset() method of node and edge h5py files. It works with MPI as well, although I couldn't write a test case for using MPI. I set 'gzip' (level 4) as the default compression method.

Compression ratio is about 1:5 (if individual weights are not stored in the h5 file) or 1:2 (if weights are stored). Regardless of the compression algorithms (none, gzip, lzf), it does not seem to affect the execution time of the simulation much.

This change should be backward compatible, as you do not need to specify the compression method when h5 files are read.

shixnya commented 1 year ago

Thanks, Kael! I made changes to the docstring. I'll merge this to develop.