trixi-framework / Trixi.jl

Trixi.jl: Adaptive high-order numerical simulations of conservation laws in Julia
https://trixi-framework.github.io/Trixi.jl
MIT License
505 stars 98 forks source link

Print global number of cells and dofs #1865

Closed benegee closed 1 month ago

benegee commented 4 months ago

Resolves #1616

github-actions[bot] commented 4 months ago

Review checklist

This checklist is meant to assist creators of PRs (to let them know what reviewers will typically look for) and reviewers (to guide them in a structured review process). Items do not need to be checked explicitly for a PR to be eligible for merging.

Purpose and scope

Code quality

Documentation

Testing

Performance

Verification

Created with :heart: by the Trixi.jl community.

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 93.88464% with 88 lines in your changes are missing coverage. Please review.

Project coverage is 96.11%. Comparing base (909abb4) to head (bded959). Report is 58 commits behind head on main.

Files Patch % Lines
...ation/paired_explicit_runge_kutta/methods_PERK2.jl 85.44% 23 Missing :warning:
src/callbacks_step/analysis_surface_integral_2d.jl 91.04% 12 Missing :warning:
src/equations/linearized_euler_1d.jl 76.60% 11 Missing :warning:
src/equations/linearized_euler_3d.jl 94.74% 5 Missing :warning:
src/Trixi.jl 0.00% 4 Missing :warning:
src/meshes/structured_mesh_view.jl 91.30% 4 Missing :warning:
src/meshes/t8code_mesh.jl 96.04% 4 Missing :warning:
src/solvers/dgsem_tree/subcell_limiters.jl 88.89% 4 Missing :warning:
src/callbacks_step/analysis.jl 86.96% 3 Missing :warning:
...em/elixir_linearizedeuler_characteristic_system.jl 93.10% 2 Missing :warning:
... and 10 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1865 +/- ## ========================================== - Coverage 96.30% 96.11% -0.19% ========================================== Files 440 460 +20 Lines 35793 36913 +1120 ========================================== + Hits 34470 35478 +1008 - Misses 1323 1435 +112 ``` | [Flag](https://app.codecov.io/gh/trixi-framework/Trixi.jl/pull/1865/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=trixi-framework) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/trixi-framework/Trixi.jl/pull/1865/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=trixi-framework) | `96.11% <93.88%> (-0.19%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=trixi-framework#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

JoshuaLampert commented 3 months ago

What about other mesh types like the TreeMesh? Does it print the local or global number of cells (see here)?

ranocha commented 3 months ago

The TreeMesh replicates all cell info on all ranks. Thus, it prints the global info.

sloede commented 1 month ago

@benegee Please note that you should also adapt the output of the AMR output, probably in these three functions: https://github.com/trixi-framework/Trixi.jl/blob/8a9fc7baeca9807de185592d5cb8d60040a24f09/src/callbacks_step/analysis.jl#L494 https://github.com/trixi-framework/Trixi.jl/blob/8a9fc7baeca9807de185592d5cb8d60040a24f09/src/callbacks_step/analysis.jl#L520 https://github.com/trixi-framework/Trixi.jl/blob/8a9fc7baeca9807de185592d5cb8d60040a24f09/src/callbacks_step/analysis.jl#L554

Otherwise we get a global element count but only rank-0 information on AMR, which is bound to cause confusion IMHO

benegee commented 1 month ago

True! I realized this in the meantime as well, but have not finished the MPI syncing of element counts.

sloede commented 1 month ago

True! I realized this in the meantime as well, but have not finished the MPI syncing of element counts.

Optimally, you'll use an implementation that only requires a single additional MPI_Reduce call.

benegee commented 1 month ago

Summary

Global number of elements and dofs

nodofsglobal and nelementsglobal are now used in the Base.show output of semidiscretizations and meshes, and of the AnalysisCallback.

For DG, nodofsglobal already relied on nelementsglobal:

https://github.com/trixi-framework/Trixi.jl/blob/76719a85e0226ba9883dcf3817c6afe8daffb3f6/src/solvers/dg.jl#L461-L463

and nelementsglobal was already MPI aware:

https://github.com/trixi-framework/Trixi.jl/blob/76719a85e0226ba9883dcf3817c6afe8daffb3f6/src/solvers/dg.jl#L520-L522

For DGMulti, ndofsglobal was already MPI aware as well:

https://github.com/trixi-framework/Trixi.jl/blob/76719a85e0226ba9883dcf3817c6afe8daffb3f6/src/callbacks_step/analysis_dgmulti.jl#L188-L194

I now added nelementsglobal in analogy.

New solver types would now have to implement ndofsglobal and nelementsglobal.

Global number of elements per level

The local element numbers per level are now summed up across all ranks before being printed on rank 0. In order to not having to synchronize the minimum and maximum element levels in advance, I took this information from the AMRController. This currently works but would require new controllers to provide the same information. If you see a better solution, please give me a hint.

sloede commented 1 month ago

Thanks a lot for tackling this @benegee! This makes Trixi.jl much more usable in parallel 💪