search

cb-geo / mpm

CB-Geo High-Performance Material Point Method
https://www.cb-geo.com/research/mpm
Other
243 stars 83 forks source link

Dynamic load balancing results in incorrect results #626

Closed kks32 closed 4 years ago

kks32 commented 4 years ago

Describe the bug Dynamic load balancing results in incorrect results.

To Reproduce Steps to reproduce the behavior: Run simulation with load balancing turned on.

Expected behavior Results should be the same as without dynamic load-balancing.

Screenshots At step 0

image

After rebalancing at 50,000 steps

image

Additional context The issue happens both with HALO_EXCHANGE on and off.

kks32 commented 4 years ago

The initial distribution of particles and boundary cells image

After the first load balancing image

When a previously ghost boundary becomes an internal part of the new boundary, particles are not transferred. This seems to be the main issue

kks32 commented 4 years ago

If particles are on the boundary cells before domain decomposition and end up in another boundary cell of a different partition it is OK.

However, if the particles were originally in the boundary cells, and in the new partition are withing another partition, then these particles are left out. This is a critical problem. When we transfer the particles, we only transfer particles between boundary zones, this is OK when we have a fixed mesh. But, if the boundary change dramatically we have to be communicate to regions within the boundary too.