Grafana dashboard for Iter8 experiments

[Alan-Cha]

• [x] #1537
• [x] #1538
  • [x] #1547
  • [x] #1548
• [x] #1539
  • [x] #1549
  • [x] #1550
  • [x] #1551
  • [x] #1552
• [x] #1553
• [x] #1554
• [x] #1555
• [x] #1556
• [x] #1557
• [x] #1558
• [x] #1541
• [x] #1580

Additional (optional) related issues

• [ ] #1524
• [ ] #1526
• [ ] #1525
• [x] #1566
  • [x] #1568
• [x] #1567
• [x] #1569
• [x] #1570

[kalantar]

Initial list of more specific items that need to be addressed:

• Design http performance test dashboard for Grafana
• Design grpc performance test dashboard for Grafana
• Metrics service API to retrieve http/grpc data for Grafana
• Storage client design to store http/grpc data
  • namespace-group::helm-revision::loop :: endpoint —> fortio (http)
  • namespace-group::helm-revision::loop :: endpoint —> ghz (grpc)
  • namespace-name::endpoint
• Update http task to write fortio results to storage client
  • Question: via metrics service? (ABn writes directly to storage client)
  • Question: implement via Driver implementation for this?
  • Question: modify ExperimentSpec.Insights or just write directly (delete Insights). MODIFY to have fortio/ghz sections (delete other sections)
• Update grpc task to write ghz results to storage client
  • Questions: same as for http task
• Other information in ExperimentResults (also store in metrics store)? YES -- all result
• Question: Move experiment spec to storage client ? NO
• Or have spec in Kubernetes (secret) and result in storage client. YES. Mount secret in job. Read it (file driver).
• delete cronjob
• Remove assess task
• Remove html/text reporters
• Remove iter8 k report command
• Remove iter8 k assess command
• Keeping: tasks -- http, grpc, readiness, notify; remove custommetrics, assess, abnmetrics
• keeping: cmd -- launch, run delete, controllers; remove report, assess, abn, log
• even more radical, remove lauch, delete (use helm). keep run, controllers; iter8 becomes "internal" only
• documentation changes
  • remove custommetrics, looped experiments, cronjob, slos, rewards, install, autox etc.
  • consider renaming group to name
  • remove autox

Implementation Steps:

• write user story / tutorial behavior
• dashboards
• storage design
• metrics http PUT
• metrics http fortio GET
• metrics grpc ghz GET
• metric serivce driver implementation
• experiment spec/results changes
• remove unused pieces

[kalantar]

User Story

1. User runs experiment namespace/name
   • helm install (or helm update) creates (updates) experiment spec
   • experiment spec (secret) is mounted to a job that runs iter8 k run
   • job reads spec using file driver from mounted secret
   • tasks run fortio/ghz and results are written to experiment result (Insight)
   • job PUTs experiment result to metrics service
   • metrics serice writes it to the storage client
2. (assume user has Grafana) User opens appropriate Grafana dashboard (http/grpc) specifying experiment name
   • if necessary user will need to define data source(s)
   • if necessary user will need to import dashboard definition
3. User inspects experiment result

[kalantar]

@Alan-Cha I think the last three items you've listed as "optional" are really required. We need to decide on strategy and implement.

[kalantar]

released in v0.16