HIP support

[llaniewski]

This PR introduces preliminary support for compilation with HIP.

To compile for the ROCM architecture, use

make configure
./configure --enable-hip

The whole code is compiled in this case with hipcc (nothing with mpic++), which means all library paths have to be discovered.

FYI: @mdzik @Me-Basha

Notes

This is preliminary.

TODO

• Tuning and testing the different warp size. Everything now is made for 32
• Moving to a more reasonable naming convention, as currently everything has "Cuda" in it.

[codecov-commenter]

Codecov Report

Base: 32.54% // Head: 32.34% // Decreases project coverage by -0.20% :warning:

Coverage data is based on head (0afdd91) compared to base (9262732). Patch coverage: 40.00% of modified lines in pull request are covered.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## develop #376 +/- ## =========================================== - Coverage 32.54% 32.34% -0.21% =========================================== Files 154 154 Lines 6422 6425 +3 =========================================== - Hits 2090 2078 -12 - Misses 4332 4347 +15 ``` | Flag | Coverage Δ | | |---|---|---| | d2q9 | `30.28% <40.00%> (-0.22%)` | :arrow_down: | | d2q9_bc | `26.51% <30.00%> (-0.21%)` | :arrow_down: | | d2q9_kuper | `27.37% <30.00%> (-0.21%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO) | Coverage Δ | | |---|---|---| | [src/Sampler.cpp.Rt](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL1NhbXBsZXIuY3BwLlJ0) | `10.20% <0.00%> (ø)` | | | [src/SyntheticTurbulence.cpp](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL1N5bnRoZXRpY1R1cmJ1bGVuY2UuY3Bw) | `8.04% <0.00%> (ø)` | | | [src/cross.cu](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL2Nyb3NzLmN1) | `80.48% <ø> (-9.76%)` | :arrow_down: | | [src/cross.h](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL2Nyb3NzLmg=) | `80.00% <ø> (-20.00%)` | :arrow_down: | | [src/cuda.cu.Rt](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL2N1ZGEuY3UuUnQ=) | `83.33% <ø> (ø)` | | | [src/main.cpp](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL21haW4uY3Bw) | `68.85% <ø> (ø)` | | | [src/Lattice.cu.Rt](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL0xhdHRpY2UuY3UuUnQ=) | `45.49% <33.33%> (-1.63%)` | :arrow_down: | | [src/ZoneSettings.h](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL1pvbmVTZXR0aW5ncy5o) | `32.94% <50.00%> (ø)` | | | [src/LatticeContainer.h.Rt](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL0xhdHRpY2VDb250YWluZXIuaC5SdA==) | `83.33% <100.00%> (ø)` | | | [src/LatticeContainer.inc.cpp.Rt](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO#diff-c3JjL0xhdHRpY2VDb250YWluZXIuaW5jLmNwcC5SdA==) | `60.67% <100.00%> (ø)` | | | ... and [5 more](https://codecov.io/gh/CFD-GO/TCLB/pull/376?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO) | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=CFD-GO)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[llaniewski]

@mdzik I introduced some changes to the hip version which reduce the number of memcpy-s done while importing geometry and exporting quantities. This was a major bottleneck on MI150 (don't know about MI250)

[mdzik]

I will benchmark it on LUMI today and let you know. There are also bottlenecks in the previous version of the branch on MI250s, so we will see. It is also advised to use GPU pointers in MPI communication - do you think its doable?

Pilot phase was extended till 8 or 10th of january, we could schedule some profiling time if you have some?

pon., 5 gru 2022, 07:55 użytkownik Łukasz Łaniewski-Wołłk < @.***> napisał:

@mdzik https://github.com/mdzik I introduced some changes to the hip version which reduce the number of memcpy-s done while importing geometry and exporting quantities. This was a major bottleneck on MI150 (don't know about MI250)

— Reply to this email directly, view it on GitHub https://github.com/CFD-GO/TCLB/pull/376#issuecomment-1336828660, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3A5R5LMTT6AIY4NRDRINTWLWGVDANCNFSM5G7M6VSQ . You are receiving this because you were mentioned.Message ID: @.***>