Benchmarking CTL Runtime

ArdanaLabs / dUSD

5 stars 3 forks source link

Benchmarking CTL Runtime #296

Open epiro-ps opened 2 years ago

epiro-ps commented 2 years ago

We want to benchmark the CTL runtime services to figure out the appropriate infrastructure to provision. Here are some ideas about what to benchmark. We can modify these to more appropriately define the task.

CTL Runtime
- Parallel connections
- Round trip HTTP for all exposed API calls
- Load tolerance
Ogmios
- Chain index synchronization time
Ogmios datum cache
- Parallel connections
- DB connection latency
- Round trip response time
- Load tolerance
Cardano Node
- Parallel connections
- Load tolerance
Postgres Database
- Read performance
- Write performance

epiro-ps commented 2 years ago

Assigned you for comment @yusdacra. The task definition here is malleable.

yusdacra commented 2 years ago

What would needed to be done for these? Would the use of NixOS tests (which run on a VM, we already do tests for the CTL runtime components) be viable for benchmarking, so we can parametrize and create these benchmarks from Nix? I'm not sure what would be used for the benchmarking, so I'm just throwing ideas here.

epiro-ps commented 2 years ago

I think we can hold off on digging into this until we get an answer on whether or not we can get access to existing CTL infrastructure. If so, we just have to ensure the provider properly benchmarked the services and can ensure reliability, scalability, durability, availability, and all the other abilities :musical_note: :musical_note:

https://github.com/ArdanaLabs/dUSD/issues/298