mec059
commented
2 years ago Hello, thank you for your feedback.
The output is to the sequential file, but we again use parallel file I/O from the MPI library (each process stores only a part of the output file). PVTU is supported by Paraview, but it contains many small files that we want to avoid (you have to duplicate shared nodes, massive parallel runs produce thousands of files and it is much easier to manipulate with a single file only, etc..).
Regarding the domain decomposition tool. The majority of tools have mesh dual graph as the input (vertices of dual represent elements and there is an edge between vertices if elements share a face). In our tool, the dual graph is built in parallel and is distributed among all MPI processes (again each process has only a part of the graph). The distributed dual graph can be directly used by ParMETIS. If you want to use METIS (sequential version of ParMETIS), we gather the dual graph to the root and scatter the METIS output back to all processes and redistribute the mesh in parrallel. I.e., METIS is sequential but the rest of the workflow is still parallel. We have this option since METIS has some settings that are not available in ParMETIS (e.g., keep domains contiguous).
I am aware of meshio tool, but it is a sequential tool only. In our approach, we have to be able to start parsing of file from an arbitrary position. Hence, parsers are significantly different. More about parallel parsing can be found here (https://www.sciencedirect.com/science/article/pii/S0965997822000138).
The library is tested, but the test suite is not a part of this repository.
Hi! This is a cool project with an interesting approach, thanks for presenting it on HPCSE. I've got some feedback after looking into the paper and trying out the library.
First, there is a famous project called meshio that does sequential input/output for many mesh formats. It might be worth reconsidering the name of the project to distinguish it, e.g. by including "par" in the name.
You can look into the meshio code for inspiration when implementing more parallel parsers/writers. It would be nice to also include a test suite to fully test the functionality of the implemented parsers on small meshes.
Getting deeper, I have a question regarding output. The README says the output is in "sequential form" to allow simple visualization of the results. How is this done exactly? Is the output performed sequentially or do you somehow manage parallel writing into a single file? Did you consider output into a parallel/partitioned format like PVTU? That would be most likely more efficient and it would also allow simple visualization since the format can be natively opened in Paraview, for example.
Regarding sequential partitioners mentioned in the README, how do you use metis or scotch on a distributed mesh? Do you need to assemble the global mesh on a single MPI rank and redistribute again?