Question/Recommendations on writing vectors/matrices to file.

[lungsi]

Hi (anyone),

I was wondering what is the preferred/recommended approach of writing neanderthal generated vectors and matrices into a file, say, an HDF5 file. What do users think of clj-hdf5?

Regards. LN

PS: I would like to read the values (vectors or matrices) for visualization (plot).

[blueberry]

HDF5 is currently out of the scope of the library. You can map vectors to a MappedByteBuffer and write the binary in a file directly, or access the contents of the vectors and write your own serialization to whatever format you desire. For example, @katox wrote his solution and published it as a stand-alone library https://github.com/katox/neanderthal-stick/tree/master/src/neanderthal_stick

Since Neanderthal supports transfer from and to typical Clojure/Java collections, clj-hdf5 will probably work out of the box, but I don't think that the performance would be anything near the built-in map-file function, or neanderthal-stick. Of course, if you want hdf5 specifically, then I guess that your main concern is compatibility with the format, and not the performance of that read/write operation.

If you only need to write and read your data in a file as fast as possible, map-file is the right function.