Track and report unpack performance

I added a simple server.unpack-perf metadata, which is a JSON block like {"min": <seconds>, "max": <seconds>, "count": <unpack_count>}, and then played with the report generator to get some statistics.

I also wrote a report of the Audit table contents to summarize the operations, statuses, and users involved in the Pbench Server.

The sample below is for a runlocal, with a few small-ish tarballs. The big catch in deploying this would be that none of the existing datasets will have server.unpack-perf until they're unpacked again, which somewhat reduces the value of the statistics until they get unpacked again (e.g., for TOC or visualize).

Nevertheless, I figured I might as well post it for consideration. Some of the statistics (and how they're calculated and/or represented) are no doubt arguable; but I enjoyed seeing the numbers anyway. 😆

Cache report:
  7 datasets currently unpacked, consuming 51.7 MB
  7 datasets have been unpacked a total of 7 times
  The least recently used cache was referenced today, fio_rw_2018.02.01T22.40.57
  The most recently used cache was referenced today, trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38
  The smallest cache is 307.2 kB, linpack_mock_2020.02.28T19.10.55
  The biggest cache is 19.6 MB, trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38
  The worst compression ratio is 22.156%, uperf_rhel8.1_4.18.0-107.el8_snap4_25gb_virt_2019.06.21T01.28.57
  The best compression ratio is 96.834%, pbench-user-benchmark_example-vmstat_2018.10.24T14.38.18
  The fastest cache unpack is 0.014 seconds, linpack_mock_2020.02.28T19.10.55
  The slowest cache unpack is 0.084 seconds, trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38
  The fastest cache unpack streaming rate is 233.226 Mb/second, trafficgen_basic-forwarding-example_tg:trex-profile_pf:forwarding_test.json_ml:5_tt:bs__2019-08-27T14:58:38
  The slowest cache unpack streaming rate is 22.228 Mb/second, linpack_mock_2020.02.28T19.10.55
  1 datasets have no unpacked size, 1 are missing reference timestamps, 0 have bad size metadata
  1 datasets are missing unpack metric data, 0 have bad unpack metric data
  1 datasets are missing unpack performance data
Audit logs:
  138 audit log rows for 69 events
  0 unterminated root rows, 0 unmatched terminators
  Status summary:
                   BEGIN         69
                 SUCCESS         68
                 FAILURE          1
  Operation summary:
                template         36
                  upload          9
                   cache          7
                   index          6
                  apikey          1
                  update         10
  Object type summary:
                TEMPLATE         36
                 DATASET         32
                 API_KEY          1
  Users summary:
              BACKGROUND         49
                  tester         18
               testadmin          2

distributed-system-analysis / pbench

Track and report unpack performance #3610