PTS insights

[marcemq]

Get insights about PTS execution based on Clear Linux OS.

[marcemq]

Installation

To install PTS bundle in a Clear Linux based OS, execute:

$ sudo swupd bundle-add phoronix-test-suite

NOTE: most of the basic PTS commands gets some time to get executed, ~ 1min even when asking for PTS help or version, which seems to be normal.

PTS directory

All PTS related (directories, tests execution, installed tests ,etc) are placed at .phoronix-test-suite , i.e.:

$ ls -la .phoronix-test-suite
total 64
drwxr-xr-x 11 pkb pkb 4096 Jan 20 15:49 .
drwxr-x---  4 pkb pkb 4096 Jan 20 16:48 ..
-rw-------  1 pkb pkb 7310 Jan 20 15:49 core.pt2so
drwxr-xr-x  2 pkb pkb 4096 Jan 20 15:37 download-cache
-rw-r--r--  1 pkb pkb 1014 Jan 20 15:42 graph-config.json
drwxr-xr-x  3 pkb pkb 4096 Jan 20 15:45 installed-tests
drwxr-xr-x  2 pkb pkb 4096 Jan 20 15:37 modules
drwxr-xr-x  2 pkb pkb 4096 Jan 20 15:37 modules-data
drwxr-xr-x  6 pkb pkb 4096 Jan 20 15:39 openbenchmarking.org
-rw-r--r--  1 pkb pkb  161 Jan 20 15:49 phoronix-test-suite-benchmark.log
drwxr-xr-x  6 pkb pkb 4096 Jan 20 15:37 test-profiles
drwxr-xr-x  3 pkb pkb 4096 Jan 20 15:47 test-results
drwxr-xr-x  4 pkb pkb 4096 Jan 20 15:37 test-suites
-rw-r--r--  1 pkb pkb 3549 Jan 20 15:38 user-config.xml
drwxr-xr-x  2 pkb pkb 4096 Jan 20 15:37 xsl

[marcemq]

PTS execution

To execute a PTS benchmark, let's say openssl benchmark with default PTS env variables execute:

$ phoronix-test-suite benchmark openssl

The first time, it'll installed the test suite and its requirements so it'll take more time to be executed, and for every execution, you'll be asked the next questions:

Would you like to save these test results (Y/n): Y
Enter a name for the result file: my-first-openssl
Enter a unique name to describe this test run / configuration: Openssl execution to be compared

OpenSSL 1.1.1:
    pts/openssl-1.11.0
    Test 1 of 1
    Estimated Trial Run Count:    3
    Estimated Time To Completion: 2 Minutes [22:03 UTC]
        Started Run 1 @ 22:02:13
        Started Run 2 @ 22:02:37
        Started Run 3 @ 22:03:02

    RSA 4096-bit Performance:
        595.1
        594.7
        593

    Average: 594.3 Signs Per Second
    Deviation: 0.19%

    Do you want to view the text results of the testing (Y/n): n
    Would you like to upload the results to OpenBenchmarking.org (y/n): n

And the output files are placed at .phoronix-test-suite/test-results/:

$ tree .phoronix-test-suite/test-results
test-results/
├── my-first-openssl
│   ├── composite.xml
│   ├── index.html
│   ├── result-graphs
│   │   ├── 1.svg
│   │   ├── 1_table.svg
│   │   ├── overview.svg
│   │   └── systems.svg
│   └── system-logs
│       └── Testing openssl BM at NUC
│           ├── cc
│           ├── cmdline
│           ├── cpuinfo
│           ├── dmesg
│           ├── environment-variables
│           ├── glxinfo
│           ├── lscpu
│           ├── lsmod
│           ├── lspci
│           ├── mdstat
│           ├── meminfo
│           ├── modules
│           ├── mounts
│           ├── uname
│           └── version

NOTE: the results can be viewed on any web browser too by loading the index.html file.

[marcemq]

Comparing results

In order to compare benchmarks results execution:

1. Identify the ID of the benchmark executed that will be considered as base benchmark i.e.: my-first-openssl

2. Execute PTS again using such ID, PTS will identify the test suite and make the comparison automatically:

   
   $ phoronix-test-suite benchmark my-first-openssl
   ...
   Current Test Identifiers:
   - Testing openssl BM at NUC
   
   Enter a unique name to describe this test run / configuration: openssl-comp

If desired, enter a new description below to better describe this result set / system configuration under test. Press ENTER to proceed without changes.

Current Description: Intel Core i7-7567U testing with a Intel NUC7i7BNB (BNKBL357.86A.0062.2018.0222.1644 BIOS) and i915drmfb on Clear Linux OS 32100 via the Phoronix Test Suite.

New Description: Comparison between benchmark executions

OpenSSL 1.1.1: pts/openssl-1.11.0 [RSA 4096-bit Performance] Test 1 of 1 Estimated Trial Run Count: 3 Estimated Time To Completion: 2 Minutes [17:09 UTC] Started Run 1 @ 17:08:34 Started Run 2 @ 17:08:58 Started Run 3 @ 17:09:22

RSA 4096-bit Performance:
    595.3
    592.8
    594

Average: 594.0 Signs Per Second
Deviation: 0.21%

Signs Per Second > Higher Is Better
Testing openssl BM at NUC . 592.9 |==================================================================================================
openssl-comp .............. 594.0 |==================================================================================================

Do you want to view the text results of the testing (Y/n): n
Would you like to upload the results to OpenBenchmarking.org (y/n): n

After the execution, the compared results will be placed at the same test results directory, on which the graphs will be updated too:

$ tree test-results/my-first-openssl/ test-results/my-first-openssl/ ├── composite.xml ├── index.html ├── result-graphs │   ├── 1.svg │   ├── 1_table.svg │   ├── overview.svg │   └── systems.svg └── system-logs ├── openssl-comp │   ├── cc │   ├── cmdline │   ├── cpuinfo │   ├── dmesg │   ├── environment-variables │   ├── glxinfo │   ├── lscpu │   ├── lsmod │   ├── lspci │   ├── mdstat │   ├── meminfo │   ├── modules │   ├── mounts │   ├── uname │   └── version └── Testing openssl BM at NUC ├── cc ├── cmdline ├── cpuinfo ├── dmesg ├── environment-variables ├── glxinfo ├── lscpu ├── lsmod ├── lspci ├── mdstat ├── meminfo ├── modules ├── mounts ├── uname └── version

4 directories, 36 files

[marcemq]

The results for the execution and comparison above are attached: my-first-openssl.zip