search

maxhodak / keras-molecules

Autoencoder network for learning a continuous representation of molecular structures.
MIT License
519 stars 146 forks source link

Enable light compression on h5 files #36

Closed dakoner closed 7 years ago

dakoner commented 7 years ago

The h5 files are being written without compression. This makes them really large, wasting disk space. It might have some runtime costs (usually it's faster to read/write compressed data into the CPU than uncompressed data). I suggest using a fairly light compression, it won't take much time to encode or decode and will still compact things significantly.

pechersky commented 7 years ago

Given that we have generator-based methods, I think this issue is fixed.

liamnaka commented 7 years ago

It could still be useful for the encoded h5 files, which use a lot of storage. Although given the high dimensionality of the latent space, storing hundreds of thousands of encoded vectors may not be feasible.