jpkenny
commented
2 years ago Hi,
We provide quite a bit of detail on how we have obtained these parameters for real systems in:
Kenny, J. P., Sargsyan, K., Knight, S., Michelogiannakis, G., & Wilke, J. J. (2018, June). The pitfalls of provisioning exascale networks: a trace replay analysis for understanding communication performance. In International Conference on High Performance Computing (pp. 269-288). Springer, Cham.
...yes, it's a lot of work to do it well.
I would suggest using the parameters from that work as a starting point:
| Parameter | Max likelihood(^{a}) | Prior range | Type |
|---|---|---|---|
| Injection bandwidth (GB/s) | 13.04 | 8.0–16.0 | Network |
| Link bandwidth (GB/s) | 12.47 | 10.0–15.0 | Network |
| Memory bandwidth (GB/s) | 11.20 | 8.0–15.0 | System software |
| Post Header Delay (us) | 0.36 | 0.1–1.5 | System software |
| Post RDMA Delay (us) | 0.88 | 0.5–2.0 | System software |
| RDMA Pin Latency (us) | 5.43 | 1.0–7.0 | System software |
| RDMA Pin Delay Per Page (ns) | 50.50 | 1.0–100.0 | System software |
| Hop Latency(^{b}) (ns) | 100 | n/a | n/a |
chenglin-li
New Issue for sst-macro
1 - Detailed description of problem or enhancement when using SSTMacro for simulation, there are many parameters need to configure in *.ini file, some parameters are easy to get, like CPU frequency, memory size, network bandwidth, but some parameters are difficult to configure, like
mpi { post_header_delay = ? post_rdma_delay = ? rdma_pin_latency = ? rdma_page_delay = ? }
do you have some testing suites or program which can be used to test on actual hardware to get these parameters, like mpi parameters above?
we run osu micro benchmark on actual hardware, and get testing result, then using SSTMacro to simulate them. for actual hardware, it is easy to get parameters like cpu, memory,etc. but it is difficult to get mpi parameters above.
could you please help? thanks.