search

Igalia / snabb

Snabb Switch: Fast open source packet processing
Apache License 2.0
47 stars 5 forks source link

Fix release benchmarks to always specify CPU #1235

Closed wingo closed 4 years ago

wingo commented 4 years ago

FWIW, on snabb1 I run benchmarks like this:

sudo SNABB_LWAFTR_CPU0=2 SNABB_LWAFTR_CPU1=3 \
  SNABB_LOADTEST_CPU0=8 SNABB_LOADTEST_CPU1=9 \
  SNABB_PCI0=02:00.0 SNABB_PCI1=82:00.0 \
  SNABB_PCI2=02:00.1 SNABB_PCI3=82:00.1 \
  SNABB_PCI4=03:00.0 SNABB_PCI5=83:00.0 \
  SNABB_PCI6=03:00.1 SNABB_PCI7=83:00.1 \
  /home/wingo/snabb/src/program/lwaftr/tests/release-benchmarks/release-benchmarks.sh
wingo commented 4 years ago

Example output:

>> Compiling configurations (may take a while)
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr2.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr2.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr2.conf: compiled configuration is up to date.
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr3.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr3.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr3.conf: compiled configuration is up to date.
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr4.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr4.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr4.conf: compiled configuration is up to date.
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr5.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr5.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr5.conf: compiled configuration is up to date.
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr6.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr6.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr6.conf: compiled configuration is up to date.
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr7.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr7.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr7.conf: compiled configuration is up to date.
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.conf: compiled configuration is up to date.
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.conf: compiled configuration is up to date.
>> Running loadtest: 1 instance, 2 NIC interface (log: /tmp/tmp.VFSfxP60fE/tmp.phyYvNb3Gy)
2019-10-28 09:29:07: WARN: failed to handle alarm op raise_alarm
2019-10-28 09:29:37: WARN: failed to handle alarm op raise_alarm
2019-10-28 09:30:07: WARN: failed to handle alarm op raise_alarm
>> Success: 7.74
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr2.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr2.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr2.conf: compiled configuration is up to date.
>> Running loadtest: 1 instance, 2 NIC interfaces (from config) (log: /tmp/tmp.VFSfxP60fE/tmp.8KrdsDZA9o)
>> Success: 4.999
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr.conf: compiled configuration is up to date.
>> Running loadtest: 1 instance, 1 NIC (on a stick) (log: /tmp/tmp.VFSfxP60fE/tmp.YNlgI3klLY)
>> Success: 9.98
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr3.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr3.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr3.conf: compiled configuration is up to date.
>> Running loadtest: 1 instance, 1 NIC (on-a-stick, from config file) (log: /tmp/tmp.VFSfxP60fE/tmp.TCWCu3KPpZ)
>> Success: 9.999
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr4.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr4.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr4.conf: compiled configuration is up to date.
>> Running loadtest: 2 instances, 2 NICs (from config) (log: /tmp/tmp.VFSfxP60fE/tmp.JHf2plrVMQ)
>> Running loadtest 2: 2 instances, 2 NICs (from config) (log: /tmp/tmp.VFSfxP60fE/tmp.lyJu40JC1k)
2019-10-28 09:33:15: WARN: failed to handle alarm op raise_alarm
>> Success: 4.999
>> Success: 7.46
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr5.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr5.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr5.conf: compiled configuration is up to date.
>> Running loadtest: 2 instances, 1 NIC (on a stick, from config) (log: /tmp/tmp.VFSfxP60fE/tmp.XXZ2lWuQYC)
>> Running loadtest 2: 2 instances, 1 NIC (on a stick, from config) (log: /tmp/tmp.VFSfxP60fE/tmp.eZcANMWzW5)
>> Success: 9.999
>> Success: 4.999
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr6.conf: loading compiled configuration from /nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr6.o
/nix/store/x1s3al6vza330j8n25c03da63j436knw-lwaftr-dataset/lwaftr6.conf: compiled configuration is up to date.
>> Running loadtest: 1 instance, 1 NIC, 2 queues (log: /tmp/tmp.VFSfxP60fE/tmp.xCmz081OmT)
>> Success: 9.999
>> Not running test for 2 instances and 4 queues. Missing LWAFTR CPUs 0,1,2 and 3 and/or
>> not configured at least 4 NICs.

FWIW these results are in Gbps, and 10 would be the max. 4.999 usually indicates that the initial check at 5 Gbps didn't work, for whatever reason (startup perhaps?), so the loadtest bracketed itself to [0, 5) Gbps, and then every check in that range succeeded.