search

sandialabs / LCM

Laboratory for Computational Mechanics
Other
12 stars 7 forks source link

ACE bug: runs with erosion do not work in parallel #95

Open ikalash opened 5 days ago

ikalash commented 5 days ago

If element removal rules are applied such that the simulation must be run in the “erosion mode,” the simulation can only be run in serial (on a single processor) and not in parallel (due to a limitation in the design of the solvers). Practically speaking, this means the wall clock time of the simulations can get large for large domains (several days to a couple weeks).

ikalash commented 5 days ago

@lxmota, @ebayats and I made a lot of progress on this at the ACE hackathon on Friday. Parallel cases run now with erosion on but do not give the same result in parallel as in serial (difference is something like 1% which is small but should not be). Looking into this is worthwhile.

lxmota commented 4 days ago

I have been thinking about this and I believe it's because the code that computes the boundary thinks that when the mesh is partitioned the processor boundaries are physical boundaries, hence the differences between serial and parallel runs. I thought I had corrected that a long time ago, but I'll double check that code.