Closed dakoner closed 7 years ago
Given that we have generator-based methods, I think this issue is fixed.
It could still be useful for the encoded h5 files, which use a lot of storage. Although given the high dimensionality of the latent space, storing hundreds of thousands of encoded vectors may not be feasible.
The h5 files are being written without compression. This makes them really large, wasting disk space. It might have some runtime costs (usually it's faster to read/write compressed data into the CPU than uncompressed data). I suggest using a fairly light compression, it won't take much time to encode or decode and will still compact things significantly.