search

open-power / hostboot

System initialization firmware for Power systems
Apache License 2.0
74 stars 97 forks source link

The impact of MCC memory availability on stream performance: #227

Closed Grubby0624 closed 1 year ago

Grubby0624 commented 1 year ago

https://github.com/open-power/hostboot/blob/dc29ac6ffc31eb959b5f16153a8d41bd147065d0/src/import/chips/p10/procedures/hwp/nest/p10_mss_eff_grouping.C#L224 I have configured memory at positions 0A/0B/1A/2A of a Proc (two DDIMMs are configured under MCC0, and one DDIMM is configured for "MCC" 1/2 respectively). Testing the Local stream performance of this Proc, the result is only 36683MB/s, which is much less than the theoretical value of 65%. May I ask:

  1. Is this phenomenon normal? Will the imbalanced memory configuration between MCCs affect memory stream performance?
  2. If it will affect, what is the reason for it? I previously understood that DDIMMs have their own OMI-Link, and the total bandwidth of several DDIMMs should be the sum of the bandwidth of each DDIMM
dcrowell77 commented 1 year ago

Bandwidth is based on a complicated set of parameters. At the very least to achieve maximum bandwidth you need to have the DDIMMs plugged into fully interleaved/balanced groups. There are also a lot of dependencies on LPAR/OS configuration. There are probably some IBM redbooks or something available that provide guidelines, but that is outside of my scope of knowledge (or anyone within the hostboot team).