Benchmarks: Microbenchmark - Add distributed inference benchmark cpp implementation

[yzygitzh]

Description Add distributed inference benchmark cpp implementation.

Major Revision

• Add dist_inference.cu for NVIDIA and AMD GPUs.
• Revise ROCm cmakes.

[codecov[bot]]

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (1f5031b) 86.00% compared to head (c875a81) 86.11%.

Files	Patch %	Lines
...ench/benchmarks/micro_benchmarks/dist_inference.py	86.56%	9 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #586 +/- ## ========================================== + Coverage 86.00% 86.11% +0.11% ========================================== Files 97 97 Lines 6851 6873 +22 ========================================== + Hits 5892 5919 +27 + Misses 959 954 -5 ``` | [Flag](https://app.codecov.io/gh/microsoft/superbenchmark/pull/586/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | Coverage Δ | | |---|---|---| | [cpu-python3.6-unit-test](https://app.codecov.io/gh/microsoft/superbenchmark/pull/586/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `71.81% <13.43%> (-0.15%)` | :arrow_down: | | [cpu-python3.7-unit-test](https://app.codecov.io/gh/microsoft/superbenchmark/pull/586/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `71.81% <13.43%> (-0.15%)` | :arrow_down: | | [cpu-python3.8-unit-test](https://app.codecov.io/gh/microsoft/superbenchmark/pull/586/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `72.22% <13.43%> (-0.15%)` | :arrow_down: | | [cuda-unit-test](https://app.codecov.io/gh/microsoft/superbenchmark/pull/586/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft) | `84.14% <86.56%> (+0.12%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[yzygitzh]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 3 pipeline(s).

[yzygitzh]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 3 pipeline(s).

[yzygitzh]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 3 pipeline(s).