SGpp / DisCoTec

MPI-based code for distributed HPC simulations with the sparse grid combination technique
https://sparsegrids.org/
GNU Lesser General Public License v3.0
8 stars 6 forks source link

Compression w/ LZ4 and MPI-IO #119

Closed freifrauvonbleifrei closed 10 months ago

freifrauvonbleifrei commented 10 months ago

write and read the (sparse grid) data in lz4 format: each rank in the output communicator writes its own data to a frame, as slice in the output file. The first rank, additionaly, writes a header frame containing the file offsets as very first frame. This way, when decompressing each rank will know where to read.