Trying to analyze times for different benchmarks using the matrix-free application, I realized that there were time sections missing, i.e., there was 10-15% of the total time that was not reported in the final time table. In addition, the functions to update multiphysics vectors were always being executed even if only the fluid dynamics was enabled (in these calls we convert from deal.II to Trilinos vectors), which lead to unnecessary time spent there.
Solution
Added additional time sections so that the time summary table reflects the overall time reported at the top. The changes were also applied to the matrix-based application so that they can be easily compared. The matrix-free application now checks whether other physics are enabled or not to only perform the update of the multiphysics vectors when needed.
As examples of the new timer output, this is how it would look like for the TGV case:
For lethe-fluid:
+---------------------------------------------------------+------------+------------+
| Total wallclock time elapsed since start | 16s | |
| | | |
| Section | no. calls | wall time | % of total |
+---------------------------------------------+-----------+------------+------------+
| Assemble RHS | 335 | 4.43s | 28% |
| Assemble matrix | 224 | 5.31s | 33% |
| Calculate CFL and percolate time vectors | 112 | 0.0411s | 0.26% |
| Calculate and output norms after Newton its | 224 | 0.0336s | 0.21% |
| Calculate enstrophy | 112 | 0.0717s | 0.45% |
| Calculate kinetic energy | 112 | 0.0514s | 0.32% |
| Distribute constraints after linear solve | 224 | 0.00389s | 0% |
| Read mesh and manifolds | 1 | 0.00292s | 0% |
| Set initial conditions | 1 | 7.72e-07s | 0% |
| Setup DOFs | 1 | 0.00489s | 0% |
| Setup ILU | 111 | 5.78s | 36% |
| Solve linear system | 224 | 0.254s | 1.6% |
| Write checkpoint | 1 | 0.00411s | 0% |
+---------------------------------------------+-----------+------------+------------+
For lethe-fluid-matrix-free:
+----------------------------------------------------------+------------+------------+
| Total wallclock time elapsed since start | 8.17s | |
| | | |
| Section | no. calls | wall time | % of total |
+----------------------------------------------+-----------+------------+------------+
| Assemble RHS | 338 | 0.0937s | 1.1% |
| Assemble matrix | 227 | 0.000153s | 0% |
| Calculate CFL and percolate time vectors | 112 | 0.0349s | 0.43% |
| Calculate and output norms after Newton its | 227 | 0.0309s | 0.38% |
| Calculate enstrophy | 112 | 0.0642s | 0.79% |
| Calculate kinetic energy | 112 | 0.0447s | 0.55% |
| Calculate time derivative previous solutions | 111 | 0.00199s | 0% |
| Distribute constraints after linear solve | 227 | 0.00216s | 0% |
| Evaluate non linear term and tau | 111 | 0.0124s | 0.15% |
| Read mesh and manifolds | 1 | 0.0029s | 0% |
| Set initial conditions | 1 | 6.51e-07s | 0% |
| Setup DoFs | 1 | 0.00221s | 0% |
| Setup GMG | 111 | 2.4s | 29% |
| Solve linear system | 227 | 5.46s | 67% |
+----------------------------------------------+-----------+------------+------------+
Testing
No time outputs are present in any test. However, I tested this with two of the matrix-free benchmarks (steady and transient) to verify that everything works as it should when outputting the timer per iteration or at the end of the simulation.
Documentation
The Timer section in the General, CFD and Multiphysics parameters section of the documentation was updated.
Miscellaneous (will be removed when merged)
Checklist (will be removed when merged)
See this page for more information about the pull request process.
Code related list:
[X] All in-code documentation related to this PR is up to date (Doxygen format)
[X] Lethe documentation is up to date
[X] The branch is rebased onto master
[X] Changelog (CHANGELOG.md) is up to date
[X] Code is indented with indent-all and .prm files (examples and tests) with prm-indent
Pull request related list:
[X] Labels are applied
[X] There are at least 2 reviewers (or 1 if small feature) excluding the responsible for the merge
[X] If this PR closes an issue or is related to a project, it is linked in the "Projects" or "Development" section
[ ] The PR description is cleaned and ready for merge
Description
Trying to analyze times for different benchmarks using the matrix-free application, I realized that there were time sections missing, i.e., there was 10-15% of the total time that was not reported in the final time table. In addition, the functions to update multiphysics vectors were always being executed even if only the fluid dynamics was enabled (in these calls we convert from deal.II to Trilinos vectors), which lead to unnecessary time spent there.
Solution
Added additional time sections so that the time summary table reflects the overall time reported at the top. The changes were also applied to the matrix-based application so that they can be easily compared. The matrix-free application now checks whether other physics are enabled or not to only perform the update of the multiphysics vectors when needed.
As examples of the new timer output, this is how it would look like for the TGV case:
lethe-fluid:lethe-fluid-matrix-free:Testing
No time outputs are present in any test. However, I tested this with two of the matrix-free benchmarks (steady and transient) to verify that everything works as it should when outputting the timer per iteration or at the end of the simulation.
Documentation
The
Timersection in theGeneral, CFD and Multiphysicsparameters section of the documentation was updated.Miscellaneous (will be removed when merged)
Checklist (will be removed when merged)
See this page for more information about the pull request process.
Code related list:
Pull request related list: