ddemidov
commented
2 years ago Using null-space vectors usually results in a heavier hierarchy (the matrices contain more unknowns and more non-zero elements). Looks like you are working with 3D models, so you probably are passing 6 null-space vectors. This means that each aggregate at the fine grid results in 6 coarse-grid unknowns. Using 3x3 blocks should be equivalent to 3 null-space vectors (only displacements, without rotations), and the resulting coarse grid is twice lighter.
This is confirmed by your logs: when null-space vectors are used, the first coarse grid contains 127518 DOFs, as opposed to 22107x3 = 66321 DOFs in the case when 3x3 blocks are used.
Also, the block-wise CSR format is simply more efficient storage-wise than the scalar one: your top-level matrix takes 3.36 GB with scalar backend, and 1.34 GB with block-wise one. Since most operations in amgcl are memory-bound, this should be the largest performance hit: sparse-matrix vector (SPMV) product is the most important operation, and it takes proportionally more time to perform with a heavier matrix. ILU0 relaxation is also based on a series of SPMVs.
In other words, the block-wise backend is simply more efficient, but it lacks some of the information from the null-space vectors, so sometimes it does not converge. I guess the resulting matrices still have a block-wise structure, so it should be possible to represent them using block CSR, but I am not sure yet how hard would it be to implement.
Re smoothed aggregation with null-space vectors: smoothed aggregation is standard aggregation followed by matrix smoothing (basically a matrix-matrix product), so it does take more time to compute and results in the heavier transfer operators. If you mean it takes more iterations to converge, and you are using block-wise backend, it sometimes helps to set precond.coarsening.aggr.eps_strong parameter to 0.
dokotor
Hi, I use amgcl for large linear elasticity models. The problems only use translational degrees of freedom (3 degrees of freedom per node). Models tend to calculate fastest with block 3x3 aggregation and ilu relaxation. This settings fail to converge for some models, and overall providing null-space vectors seems to help with convergence.
When using null-space vectors, the overall calculation time increases by a lot. Looking at the profiler, it seems that relaxation step is taking a lot more time. Is there any possibility of adjusting (or editing in code) ilu relaxation, so that it behaves similarly to situation, when block solve is being run? If I understand correctly, that would give me best of both routes - coarsening using null-space vectors (better convergence) and block relaxation (faster calculation).
Here are is an example from both runs.
On an unrelated note, I am also struggling with using smoothed aggregation with null-space vectors. They calculate much longer that standard aggregation, which seems counter-intuitive.