trixi-framework / Trixi.jl

Trixi.jl: Adaptive high-order numerical simulations of conservation laws in Julia
https://trixi-framework.github.io/Trixi.jl
MIT License
527 stars 105 forks source link

Use @batch reduction functionality for subcell bounds check #1888

Closed bennibolm closed 6 months ago

bennibolm commented 6 months ago

Polyester.jl supports an automatic reduction functionality with @batch since the newest version v0.7.10. The bounds check of subcell limiting requires a reduction step of deviations, which created a lot of issue due to False Sharing in the past. After some testing and discussion, I implemented a relatively ugly fix in PR #1736. With the new Polyester functionality, we can achieve a much nicer solution with approximately the same performance and scaleability.

Test within branch bennibolm/subcell-limiting and fix in bennibolm/subcell-limiting-@batch-reduction

trixi_include("../examples/tree_2d_dgsem/elixir_euler_blast_wave_sc_subcell_nonperiodic.jl",
                       initial_refinement_level = 5)

Time results of the bounds check are

Roci:       old version                 new version wit @batch reduction
            time     %tot     avg       time     %tot     avg
1 Thread:   715ms    6.9%   457μs       707ms    6.8%   452μs
2 Threads:  348ms    4.7%   223μs       341ms    4.7%   218μs
4 Threads:  177ms    3.8%   113μs       173ms    3.9%   111μs
6 Threads:  121ms    3.3%  77.1μs       121ms    3.4%  77.4μs
8 Threads:  94.3ms   3.0%  60.3μs       93.0ms   2.9%  59.5μs
10 Threads: 77.4ms   2.6%  49.5μs       76.4ms   2.6%  48.9μs
12 Threads: 68.0ms   2.5%  43.5μs       66.7ms   2.4%  42.7μs
16 Threads: 55.1ms   2.2%  35.3μs       51.9ms   2.1%  33.2μs
24 Threads: 42.8ms   1.9%  27.4μs       38.3ms   1.7%  24.5μs
github-actions[bot] commented 6 months ago

Review checklist

This checklist is meant to assist creators of PRs (to let them know what reviewers will typically look for) and reviewers (to guide them in a structured review process). Items do not need to be checked explicitly for a PR to be eligible for merging.

Purpose and scope

Code quality

Documentation

Testing

Performance

Verification

Created with :heart: by the Trixi.jl community.

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 93.75000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 96.31%. Comparing base (01032aa) to head (51ad911).

Files Patch % Lines
src/callbacks_stage/subcell_bounds_check_2d.jl 92.86% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1888 +/- ## ========================================== - Coverage 96.31% 96.31% -0.00% ========================================== Files 440 440 Lines 35797 35788 -9 ========================================== - Hits 34476 34467 -9 Misses 1321 1321 ``` | [Flag](https://app.codecov.io/gh/trixi-framework/Trixi.jl/pull/1888/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=trixi-framework) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/trixi-framework/Trixi.jl/pull/1888/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=trixi-framework) | `96.31% <93.75%> (-<0.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=trixi-framework#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.