KratosMultiphysics / Kratos

Kratos Multiphysics (A.K.A Kratos) is a framework for building parallel multi-disciplinary simulation software. Modularity, extensibility and HPC are the main objectives. Kratos has BSD license and is written in C++ with extensive Python interface.
https://kratosmultiphysics.github.io/Kratos/
Other
991 stars 244 forks source link

Alternative linear system solvers for the subspace iterations in Eigenvalue Analysis #8097

Open emiroglu opened 3 years ago

emiroglu commented 3 years ago

This was initially mentioned in #8064 here

The regular compilation of Kratos defaults to sparse_lu from Eigen library, that causes excessive memory usage in some large cases and leads to termination of the simulation. When Kratos is compiled with IntelMKL, the linear system solver defaults to the pardiso_ldlt, which can handle such big cases. Code location

I think it would make sense to have an option for selecting the linear system solver from ProjectParameters.json, so that the user is also aware of this possibility. This might also help the use of other solvers such as iterative solvers, etc.

@KratosMultiphysics/structural-mechanics

loumalouomega commented 3 years ago

You can choose the linear solver, at least for the solvers we don't do it automatically. Yes, it would be clever to add a Parameters (specially because we already have a linear solver factory)

philbucher commented 3 years ago

I think the reason the linear solver factory is not yet used is bcs the eigensolver directly uses an interface of Eigen @armingeiser @oberbichler can you confirm please?

armingeiser commented 3 years ago

It is important to have in mind that here multiple RHS are solved internally using the linear solver. As far as i know, only the direct solvers from LinearSolversApplication are currently able to factorize once, and then solve multiple RHS.

With other solvers this might get very expensive. That was the reason we removed that setting (it was available in the first implementation).

oberbichler commented 3 years ago

@philbucher no

I completely forgot that there is a switch in the JSON to use eigen_lu even when Kratos was compiled with MKL.

Anyway: If your are able to compile with MKL then pardiso_ldlt is the best choice. If you are not able to compile with MKL eigen_lu is the best choice.

If we find out that selecting another solver would bring benefits we could integrate this feature. Until then I would not spend time on this extension.

armingeiser commented 3 years ago

I think the reason the linear solver factory is not yet used is bcs the eigensolver directly uses an interface of Eigen @armingeiser @oberbichler can you confirm please?

Just looked into the code again, and you are right. The current implementation makes heavy use of Eigen types internally and it is not straight forward to allow other "Kratos"-solvers.

emiroglu commented 3 years ago

Coming from #8330 I am planning to add a linear_solvers_settings in the ProjectParameters.json for selecting the MKL solvers. Here, the start would be to make only the solvers available, which are already interfaced. The default would be pardiso_ldlt as it was before, since it is apparently the fastest one. Any suggestions and comments are appreciated...