abarret / multiphase-stokes

Solver a mixture of fluids based on IBAMR
1 stars 0 forks source link

Instrument with timers and add mean zero div #50

Closed abarret closed 1 year ago

abarret commented 1 year ago

This adds SAMRAI timers to FullFACPreconditioner and prints timer data in the multigrid target.

This also adds an option to make the RHS of the mass equation have discrete mean zero. This should fix the issue with non-convergence that @bindi-nagda was observing. See https://github.com/abarret/multiphase-stokes/issues/46#issuecomment-1627762164.

Note that you need to be very careful when reporting timer data. First, you should make sure that you run with optimizations enabled (-O2 or -O3). Second, you want to report time data averaged over several runs, so that you can minimize any background computer process interference.

In terms of reporting timer data in a paper, I'm not sure how helpful that will be. Different implementations will spend different amount of time in different places. For example, a PETSc implementation will spend time in different function calls than a MATLAB implementation. However, those two implementations should take the same number of Krylov iterations assuming they implement the same algorithm. Krylov iterations is what is reproducible, and I think that should be the metric we report in any kind of publication.

According to the timers, we spend almost all of our time in smoothError(): roughly 85%. We may be able to speed this up by switching the underlying box relaxation solver, but I'm not sure how much we will gain. One thing that might help is replacing the lapack solver with one that has known matrix sizes at runtime. The compiler might be able to vectorize and unroll the loops in this case. What will speed up computations is using a different smoother. We may be able to save a percent or so in run time if we replace the TwoFluidStaggeredStokesOperator::apply() with a Fortran implementation.