[Guardrails] Data Collection Automation - Session Generation - Githubissues

pokt-network / pocket-core

Official implementation of the Pocket Network Protocol

http://www.pokt.network

MIT License

209 stars 103 forks source link

[Guardrails] Data Collection Automation - Session Generation #1515

Open jessicadaugherty opened 1 year ago

jessicadaugherty commented 1 year ago

Objective

To measure the efficacy of pocket-score scalability efforts, we should automate the collection of key data points to inform us on the impacts of changes in controlled and isolated environments. Bonus: classify service.

Origin Document

Guardrails Projects

Guardrails Recommendations

Guardrails Presentation

Goal

Automate the collection of Session Generation data

Deliverable

Create an IAVL only script with IAVL stores (Goleveldb) to:
- Set Y Applications
- Set Servicers
- Run X Dispatch Requests in a row
- Observe htop for resources
- TimeTrack individual dispatch calls
The script should include trials with different values for Y applications and X dispatch requests (TBD)
Log the results and determine an interval at which to log this data

Non-goals / Non-deliverables

Automating the collection of additional data points

General issue deliverables

[ ] Update the appropriate CHANGELOG
[ ] Update any relevant READMEs (local and/or global)
[ ] Update any relevant global documentation & references
[ ] If applicable, update the source code tree explanation
[ ] If applicable, add or update a state, sequence or flowchart diagram using mermaid

Creator: @jessicadaugherty

jessicadaugherty commented 1 year ago

See if this data is available in Prometheus. If not, update deliverable to ensure information about duration and resources used in session generation are made available via Prometheus. Remove trials: will have real world/dynamic data/scenarios.