KratosMultiphysics / Kratos

Kratos Multiphysics (A.K.A Kratos) is a framework for building parallel multi-disciplinary simulation software. Modularity, extensibility and HPC are the main objectives. Kratos has BSD license and is written in C++ with extensive Python interface.
https://kratosmultiphysics.github.io/Kratos/
Other
991 stars 244 forks source link

[Core] Incorrect AMGCLSolver::mBlockSize #11853

Open matekelemen opened 7 months ago

matekelemen commented 7 months ago

Judging by how it's calculated, I'm assuming AMGCLSolver::mBlockSize should represent the number of DoFs per variable (or node, I'm not sure), but it seems to me that the current implementation is broken.

Can someone explain what exactly is mBlockSize supposed to be equal to on the Kratos side?

I'm running a linear structural mechanics case with quadratic elements and some MPCs. For this kind of model, the number of DoFs in a variable should be 3 (x, y, z displacement or rotation components). However, AMGCLSolver ends up with an mBlockSize of 1, which completely derails the solver to the point it doesn't even converge (hardcoding it to 3 results in convergence).

Here's how mBlockSize is calculated:

https://github.com/KratosMultiphysics/Kratos/blob/86d7a1f8fe1ffbc14029a0fb89a94bf7b2ccd2f0/kratos/linear_solvers/amgcl_solver.h#L525-L547

It appears to me that this piece of code is supposed to count how many DoFs share the same ID, and makes 2 assumptions: 1) every ID is referenced by the same number of DoFs, so it's enough to count once (the first ID) 2) DoFs sharing their IDs are consecutive in rDofSet

The first assumption seems reasonable to me, but the second does not. rDofSet is a PointerVectorSet<Dof<double>>, which means that it's sorted by the values of raw Dof pointers. I don't see how this would guarantee consecutive IDs.

Am I missing something or is this a bug?

P.S.: the solver does actually converge with a block size of 1, but more than 500% slower (for my specific model).

AlejandroCornejo commented 7 months ago

Wow, that's very interesting (as much as concerning...). I had the same idea about what is the Block size and seems that this piece of code is not workling as expected... I pinned @ddemidov and @RiccardoRossi to be sure about this

ddemidov commented 7 months ago

I am not familiar with Kratos internals well enough to understand what is the problem here, sorry.

RiccardoRossi commented 7 months ago

for (aggregation) AMG to work correctly you need blocks of dofs to be repeated for every node. let's imagine that we have dx dy dz to be the displacements

if we have

dx_1 dy_1 dz_1 dx_2 dy_2 dz_2 dx_3 dy_3 dz_3

than all of the nodes 1 2 and 3 have the same anount of dofs. 3 in this case.

however if you have for example

dx_1 dy_1 dz_1 dx_2 dy_2 dx_3 dy_3 dz_3

that is, for example if 2 does not have "dz" in the system, BlockSize will be detected as 1 as it is the only multiplier we have

the "BlockBuilderAndSolver" ensures that this property is kept. The Eliminiation builder and solver does not ...

to be clear, if you have a multidimensional problem (like structures) and the BlockSize==1 aggregation multigrid will NOT work correctly. You could still use RugenStuben for example, but aggregation based amg will not work properly

dcagritan commented 7 months ago

Do we have documentation regarding solvers in Kratos? Documentation or a wiki page that explains the parameters of the solvers for instance?

RiccardoRossi commented 7 months ago

no

El vie., 1 dic. 2023 16:21, Deniz Tanyildiz @.***> escribió:

Do we have documentation regarding solvers in Kratos? Documentation or a wiki page that explains the parameters of the solvers for instance?

— Reply to this email directly, view it on GitHub https://github.com/KratosMultiphysics/Kratos/issues/11853#issuecomment-1836297002, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5PWEI72VQZSO57KURT6RLYHHYX3AVCNFSM6AAAAAA72GF36GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZWGI4TOMBQGI . You are receiving this because you were mentioned.Message ID: @.***>