Add portal-report playbook

[ChrisSchinnerl]

PULL REQUEST

Overview

The goal of this MR is to create a report playbook which we can execute periodically to create a report for publishing via blog posts etc.

It should contain the following metrics.

Aggregates

• [x] Files
• [x] Storage

Medians (p90/p99/p999)

• [x] Downloads
• [x] Uploads
• [x] Registry reads
• [x] Registry writes

Average over all servers for

• [x] Spending breakdown

Issues Closed

Closes SKY-1290

 Example output



MSG:

TotalNumFiles:              22411690
TotalStorage:               305.4268672559383 TB
TotalStorageSpending:       1043.3375235117983 KS
TotalUploadSpending:        55.722839123054236 KS
TotalDownloadSpending:      1481.3293596239698 KS
TotalRegistryReadSpending:  244.16294079427902 KS
TotalRegistryWriteSpending: 483.3459406707132 SC
TotalSubscriptionSpending:  4.962545390430276 SC

MedianBaseSectorUpload(p99): 14336 ms
  - siasky.net:                14336 ms
  - skynetfree:                23552 ms
  - skynetpro:                 12288 ms
MedianStreamBufferRead(p99): 3840 ms
  - siasky.net:                3776 ms
  - skynetfree:                6912 ms
  - skynetpro:                 4864 ms
MedianRegistryRead(p99):     752 ms
  - siasky.net:                752 ms
  - skynetfree:                6912 ms
  - skynetpro:                 480 ms
MedianRegistryWrite(p99):    304 ms
  - siasky.net:                336 ms
  - skynetfree:                228 ms
  - skynetpro:                 336 ms

ContractSpending Totals:
  Total Allocated:         4811.695393878956 KS
  Total Spent:             2284.9992623001517 KS
    Storage:               1043.3375235117983 KS
    Uploads:               55.722839123054236 KS
    Downloads:             0 H
    FundAccount:           953.2285012614532 KS
    Fees:
      Contracts:           128.01802640918808 KS
      Txns:                104.6923719946579 KS
      Maintenance:
        GetAccountBalance: 2845172 H
        FundAccount:       1937365 H
        UpdatePriceTable:  76639018 H
  Unspent:                 6701.857499889854 KS
    Allocated:             2526.696131578804 KS
    Unallocated:           4175.16136831105 KS
  PreviousSpending:        15240.503929781675 KS

ContractSpending Averages:
  Average Total Allocated:         87.48537079779919 KS
  Average Total Spent:             41.54544113273003 KS
    Average Storage:               18.96977315475997 KS
    Average Uploads:               1.0131425295100769 KS
    Average Downloads:             0.0 H
    Average FundAccount:           17.331427295662785 KS
    Fees:
      Average Contracts:           2.3276004801670562 KS
      Average Txns:                1.9034976726301436 KS
      Maintenance:
        Average GetAccountBalance: 51730.4 H
        Average FundAccount:       35224.818181818184 H
        Average UpdatePriceTable:  1393436.690909091 H
  Average Unspent:                 121.85195454345188 KS
    Average Allocated:             45.93992966506916 KS
    Average Unallocated:           75.91202487838272 KS
  Average PreviousSpending:        277.1000714505759 KS

EphemeralAccountSpending Totals:
  Downloads:       1334.0577399560868 KS
  Uploads:         0 H (NOTE: still using legacy uploads)
  RegistryRead:    244.16294079427902 KS
  RegistryWrite:   483.3459406707132 SC
  RepairDownload:  147.271619667883 KS
  RepairUpload:    0 H (NOTE: still using legacy uploads for repairs)
  Subscriptions:   4.962545390430276 SC

EphemeralAccountSpending Averages:
  Average Downloads:       24.25559527192885 KS
  Average Uploads:         0.0 H (NOTE: still using legacy uploads)
  Average RegistryRead:    4.439326196259619 KS
  Average RegistryWrite:   8.788108012194787 SC
  Average RepairDownload:  2.677665812143327 KS
  Average RepairUpload:    0.0 H (NOTE: still using legacy uploads for repairs)
  Average Subscriptions:   0.09022809800782321 SC

Closes SKY-1295

[linear[bot]]

SKY-1290 Portal-Report Ansible Playbook

[linear[bot]]

SKY-1295 Portals report - for David's blog post

[MSevey]

Still needs a README entry