search

cloud-bulldozer / benchmark-operator

The Chuck Norris of cloud benchmarks
Apache License 2.0
282 stars 127 forks source link

Output format is different if we run uperf manully and uperf with Benchmark Operator #807

Closed vpuliyal closed 1 year ago

vpuliyal commented 1 year ago

Manually I ran uperf on POD.

sh-4.4$ uperf -v -a -R -i 1 -m /tmp/uperf.xml -P 30005
Error getting SSL CTX:1
Allocating shared memory of size 156624 bytes
Completed handshake phase 1
Starting handshake phase 2
Handshake phase 2 with 10.129.4.30
  Done preprocessing accepts
  Sent handshake header
  Sending workorder
    Sent workorder
    Sent transaction
    Sent flowop
    Sent transaction
    Sent flowop
    Sent transaction
    Sent flowop
TX worklist success  Sent workorder
Handshake phase 2 with 10.129.4.30 done
Completed handshake phase 2
Starting 1 threads running profile:stream-tcp-1024-1024-1 ...   0.00 seconds
TX command [UPERF_CMD_NEXT_TXN, 0] to 10.129.4.30
timestamp_ms:1678434982950.2659 name:Txn1 nr_bytes:0 nr_ops:0
timestamp_ms:1678434983951.2925 name:Txn1 nr_bytes:0 nr_ops:1

TX command [UPERF_CMD_NEXT_TXN, 1] to 10.129.4.30
timestamp_ms:1678434983951.3643 name:Txn2 nr_bytes:0 nr_ops:0
.
.
Sending signal SIGUSR2 to 140735581516144
called out
timestamp_ms:1678435043209.5227 name:Txn2 nr_bytes:223602540544 nr_ops:3411904

TX command [UPERF_CMD_NEXT_TXN, 2] to 10.129.4.30
timestamp_ms:1678435043209.5967 name:Txn3 nr_bytes:0 nr_ops:0
timestamp_ms:1678435043209.6050 name:Txn3 nr_bytes:0 nr_ops:0

-------------------------------------------------------------------------------------------------------------------------------
TX command [UPERF_CMD_SEND_STATS, 0] to 10.129.4.30
timestamp_ms:1678435043309.7739 name:Total nr_bytes:223602540544 nr_ops:3411905

Group Details
-------------------------------------------------------------------------------------------------------------------------------
timestamp_ms:1678435043209.6309 name:Group0 nr_bytes:13975158783 nr_ops:3411906

Strand Details
-------------------------------------------------------------------------------------------------------------------------------
timestamp_ms:1678435043209.6328 name:Thr0 nr_bytes:223602540544 nr_ops:3411907

Txn                Count         avg         cpu         max         min
-------------------------------------------------------------------------------------------------------------------------------
Txn0                   1    151.48us      0.00ns    151.48us    151.48us
Txn1              213245    276.86us      0.00ns      1.94ms     39.60us
Txn2                   1     25.38us      0.00ns     25.38us     25.38us

Flowop             Count         avg         cpu         max         min
-------------------------------------------------------------------------------------------------------------------------------
connect                1    151.31us      0.00ns    151.31us    151.31us
write            3411904     17.30us      0.00ns      1.94ms     39.53us
disconnect             1     24.96us      0.00ns     24.96us     24.96us

Netstat statistics for this run
-------------------------------------------------------------------------------------------------------------------------------
Nic       opkts/s     ipkts/s      obits/s      ibits/s
eth0       110194      109188    29.21Gb/s    57.65Mb/s
-------------------------------------------------------------------------------------------------------------------------------

Run Statistics
Hostname            Time       Data   Throughput   Operations      Errors
-------------------------------------------------------------------------------------------------------------------------------
[10.129.4.30] Success10.129.4.30       61.36s   208.24GB    29.15Gb/s      3411844        0.00
master            61.36s   208.25GB    29.15Gb/s      3411907        0.00
-------------------------------------------------------------------------------------------------------------------------------
Difference(%)     -0.00%      0.00%        0.00%        0.00%       0.00%
vpuliyal commented 1 year ago

Output of uperf with Benchmark Operator 

<?xml version=1.0?>
<profile name="stream-tcp-1024-1024-1">
          <group nthreads="1">
      <transaction iterations="1">
        <flowop type="connect" options="remotehost=$h protocol=tcp port=30001"/>
      </transaction>
      <transaction duration="120">
        <flowop type=write options="count=16 size=1024"/>
      </transaction>
      <transaction iterations="1">
        <flowop type=disconnect />
      </transaction>
  </group>
        </profile>
/usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25.11) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
2023-03-08T17:08:12Z - INFO     - MainProcess - run_snafu: logging level is INFO
2023-03-08T17:08:12Z - INFO     - MainProcess - _load_benchmarks: Successfully imported 3 benchmark modules: coremarkpro, systemd_analyze, uperf
2023-03-08T17:08:12Z - INFO     - MainProcess - _load_benchmarks: Failed to import 0 benchmark modules:
2023-03-08T17:08:12Z - INFO     - MainProcess - run_snafu: Not connected to Elasticsearch
2023-03-08T17:08:12Z - INFO     - MainProcess - wrapper_factory: identified uperf as the benchmark wrapper
2023-03-08T17:08:12Z - INFO     - MainProcess - _benchmark: Starting uperf wrapper.
2023-03-08T17:08:12Z - INFO     - MainProcess - _benchmark: Running setup tasks.
2023-03-08T17:08:12Z - INFO     - MainProcess - _benchmark: Collecting results from benchmark.
2023-03-08T17:08:12Z - INFO     - MainProcess - uperf: Collecting 1 sample of Uperf
2023-03-08T17:08:12Z - INFO     - MainProcess - process: Collecting 1 sample of command ['uperf', '-v', '-a', '-R', '-i', '1', '-m', '/tmp/uperf-test/uperf-stream-tcp-1024-1024-1', '-P', '30000']
2023-03-08T17:10:15Z - INFO     - MainProcess - uperf: Finished collecting sample 0
2023-03-08T17:10:15Z - WARNING  - MainProcess - uperf: The following params will be overwritten due to values found in workload profile name: test_type, protocol, num_threads
2023-03-08T17:10:15Z - INFO     - MainProcess - uperf: --------------------------------------------------
2023-03-08T17:10:15Z - INFO     - MainProcess - uperf: Summary result for sample : 0
2023-03-08T17:10:15Z - INFO     - MainProcess - uperf: Average byte : 282509038.8833333
2023-03-08T17:10:15Z - INFO     - MainProcess - uperf: Average ops : 275887.7833333333
2023-03-08T17:10:15Z - INFO     - MainProcess - uperf: 95%ile Latency(ms) : 7.461912948464237
2023-03-08T17:10:15Z - INFO     - MainProcess - uperf: --------------------------------------------------
2023-03-08T17:10:15Z - INFO     - MainProcess - process: Finished collecting 1 sample for command ['uperf', '-v', '-a', '-R', '-i', '1', '-m', '/tmp/uperf-test/uperf-stream-tcp-1024-1024-1', '-P', '30000']
2023-03-08T17:10:15Z - INFO     - MainProcess - uperf: Successfully collected 1 sample of Uperf.
2023-03-08T17:10:15Z - INFO     - MainProcess - _benchmark: Cleaning up
2023-03-08T17:10:15Z - INFO     - MainProcess - run_snafu: Duration of execution - 0:02:03, with total size of 28800 bytes
<?xml version=1.0?>
stale[bot] commented 1 year ago

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.