search

openucx / ucc

Unified Collective Communication Library
https://openucx.github.io/ucc/
BSD 3-Clause "New" or "Revised" License
177 stars 85 forks source link

TL/UCP: adjust barrier radix #915

Closed shimmybalsam closed 4 months ago

shimmybalsam commented 5 months ago

What

Change default radix for TL/UCP barrier to 8

Why ?

Better perf for OOB

shimmybalsam commented 5 months ago

dpu-rock (AMD) 8 nodes: <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

  | hcoll | ucc oob(radix=4) | ucc radix = 2 | ucc radix = 8 -- | -- | -- | -- | -- ppn=1 | 3.57 | 4.06 | 4.98 | 3.27 ppn=64 | 12.43 | 15.12 | 14.41 | 13.58

jazz (Intel) 8 nodes: <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

  | hcoll | ucc oob(radix=4) | ucc radix = 2 | ucc radix = 8 -- | -- | -- | -- | -- ppn=1 | 2.13 | 2.42 | 3.19 | 1.78 ppn=64 | 4.66 | 5.49 | 6.45 | 4.95