Currently LaMEM writes a lot of output files, as each processor writes their part of the grid to disk at every requested timestep. The files are in binary format but not compressed.
That is bad for most HPC systems, as their (typical) filesystems rather deal with fewer and larger files than many smaller ones. Changing this within LaMEM itself is cumbersome as it involves copying all files to rank 0. We have looked into using other fileformats such as HDF5 but i) installing that along with PETSc is (or was) tricky and ii) this still doesn't generate files that can directly be opened by Paraview, which is a must for a good user experience.
One potential workaround is by using the WriteVTK package. This package allows writing files to disk in compressed binary format, which saves space. In addition, LaMEM.jl already opens the many files for a given timestep and reads them into a single structure.
The idea is thus to write a routine that automatically processes all timesteps in a LaMEM simulation and:
Reads a given timestep with all fields.
Writes back that timestep as a single, compressed, file to the same (timestep) directory.
Generate a new *.pvtr file in the same directory, that points to the single new file, rather than to the many smaller ones.
Delete the smaller *.vtr files and rename the new *.pvtr file to have the same name as the old *.pvtr file.
As a result you should still be able to open the *.pvd file as before, but now it will have only one file per timestep rather than many smaller ones. This should load faster in paraview and will use less disk space.
By making this routine part of LaMEM.jl, you can postprocess your simulations once the simulation is done.
Currently LaMEM writes a lot of output files, as each processor writes their part of the grid to disk at every requested timestep. The files are in binary format but not compressed.
That is bad for most HPC systems, as their (typical) filesystems rather deal with fewer and larger files than many smaller ones. Changing this within LaMEM itself is cumbersome as it involves copying all files to rank 0. We have looked into using other fileformats such as
HDF5but i) installing that along with PETSc is (or was) tricky and ii) this still doesn't generate files that can directly be opened by Paraview, which is a must for a good user experience.One potential workaround is by using the WriteVTK package. This package allows writing files to disk in compressed binary format, which saves space. In addition, LaMEM.jl already opens the many files for a given timestep and reads them into a single structure.
The idea is thus to write a routine that automatically processes all timesteps in a LaMEM simulation and:
*.pvtrfile in the same directory, that points to the single new file, rather than to the many smaller ones.*.vtrfiles and rename the new*.pvtrfile to have the same name as the old*.pvtrfile.As a result you should still be able to open the
*.pvdfile as before, but now it will have only one file per timestep rather than many smaller ones. This should load faster in paraview and will use less disk space.By making this routine part of LaMEM.jl, you can postprocess your simulations once the simulation is done.