Parallel offline stage to serial online stage

[JacobLotz]

Hi all, I have been working on implementation of libROM in our own code for some time now. With help of some of you I have thus far implemented a ROM of a non-linear equation with DEIM hyper-reduction with a NURBS discretisation as a full order model. Thank you for your help so far, especially @dylan-copeland deserves a shout-out! I am currently working on a parallel version of the code. The parallel offline sampling and merge stage work, but the online stage is a bit more tricky.

I think the easiest way to perform the online stage is to convert the whole parallel system to a serial system using the class SampleMeshManager::WriteVariableSampleMap using a "to-serial" step, some sort of second merge step. Then in the online stage I can use the class SampleDOFSelector. In the "to-serial" step I have to gather the parallel basis to the root process and export the sampled FiniteElementSpace and mesh. The sampled basis is luckily already gathered to the root process.

Is there any example code using SampleDOFSelector in the online phase? I hope to use it as inspiration but I cannot find an example in examples. I am interested in:

• I think using SampleDOFSelector is quite straightforward but I am mostly interested in strategies to merge the parallel basis into a serial one. I think using SampleMeshManager can do the trick, but it is probably not the most straight forward way to do it.
• Also printing and reading the sampled FiniteElementSpace is going to be difficult I think.

Do you have suggestions for these?

[dylan-copeland]

Hi @JacobLotz, fortunately we have an example in the rom branch of Laghos for you to follow. This workflow is more complicated than any of the libROM examples. Here is an example of SampleDOFSelector:

https://github.com/CEED/Laghos/blob/b32ae04b8afaff2b8bd1cc0380342f597c66efd1/rom/laghos_rom.cpp#L2124

The regression tests illustrate parallel simulations, with serial online simulation, e.g.

rom/tests/sedov-blast/sedov-blast-time-window.sh

In this script, $LAGHOS should be something like mpirun -n 8 laghos, and $LAGHOS_SERIAL should be mpirun -n 1 laghos. Note that the romhrprep is a parallel online preprocessing step. Only the online simulation is serial. I hope this helps, but feel free to ask if some things are still unclear.

[JacobLotz]

Hi @dylan-copeland, excellent! I should be able to get something out of this. I will let you know when I need more info.

Kind regards,

Jacob