Improve the advection solver (matrix allocation, assembling, solving)

[yinghe616]

Yesterday when I run a 3d falling box test, I found that the #dof of the advection fields increased too much as the mesh refined, although it’s an expected drawback of the DG with lower order approach. However, when I went though the solver part, I think there are some parts at least we can improve to reduce the computational cost in terms of memory and time for the advection fields.

1, matrix allocation: since the linear solver always solve the advection fields sequentially in the order of temperature, first composition, second composition…, if the polynomial degree of temperature and composition are the same, then their matrix sparsities should be the same, for such case, we may can allocate the matrix only one at the initial setup and reinitialize it for each fields. This should be done for both FEM and DG? If the polynomial degree of temperature and composition are not the same, at least we can do that to multiple compositional fields case.

2, assembling: for multiple compositional fields, only assemble the matrix once for the DG solver as all the filed’s matrix are the same.

3, solve: for the DG solver, pass multiple rhs vectors as a rectangular to the linear solver and solve only once, instead of solving multiple times for each rhs vector, which I assumes the matrix -matrix multiplication will be faster than the matrix-vector computation.

4, matrix-free for advection fields: solve advection fields by using trilinos doesn’t gain too much as the velocity field does… as usually the matrices are well-conditioned symmetric matrix can be solved by CG very faster… but the assembling the matrix for the DG advection solver could be a dominant part.

Below is the first step of the 3D falling box test I was running. You can have a sense how the #dof increased as the mesh refined.

Number of active cells: 4,096 (on 5 levels) Number of degrees of freedom: 259,253 (107,811+4,913+35,937+110,592)

*\ Timestep 0: t=0 years Skipping temperature solve because RHS is zero. Solving C_1 system ... 0 iterations. Rebuilding Stokes preconditioner... Solving Stokes system... 30+7 iterations.

Number of active cells: 8,331 (on 6 levels) Number of degrees of freedom: 541,820 (230,067+10,127+76,689+224,937)

*\ Timestep 0: t=0 years Skipping temperature solve because RHS is zero. Solving C_1 system ... 0 iterations. Rebuilding Stokes preconditioner... Solving Stokes system... 30+7 iterations.

Number of active cells: 37,647 (on 7 levels) Number of degrees of freedom: 2,415,707 (1,016,379+44,066+338,793+1,016,469)

[yinghe616]

And also one way can help to improve as Sam suggested is to use DGP instead of DGQ.

[bangerth]

@yinghe616 -- instead of having this one github issue that contains lots of suggestions for individual improvements, can you open one issue for each suggestion? This makes it much simpler to track which have already been taken care of and which one are still in need of work.

If you open one issue for each item here, feel free to close this one.

[bangerth]

For reference, #1037 was one attempt to address one current short coming for DG fields.

[yinghe616]

@bangerth, just noticed your comment from yesterday. Sure~, I will open individual issue about individual improvement later and close this one. And of course, I would like to keep working on fixing those issues. :)

[gassmoeller]

This feels like a meta issue that is not well constrained (these things stay open for a long time, and nobody knows if they are already addressed). As discussed above, please open individual issues for the parts that are not yet addressed. I will close this for now.