feat: run stress test on PRs

fleek-network / ursa

Ursa, a decentralized content delivery network that is censorship-resistant and trustless without compromise on throughput and latency

https://fleek.network

Apache License 2.0

142 stars 34 forks source link

feat: run stress test on PRs #434

Open kckeiks opened 1 year ago

kckeiks commented 1 year ago

It would be useful to be able to run a stress test for the gateway on command.

Suggestion:

In a container, run a few nodes, storetheindex, and gateway.
Nodes can provide ~3 files of different sizes.
Test gateway with oha or some other tool.
Display results on PR.

Consider that we should run all participants (indexer, node, gateway) locally in test so that latency (due to its volatility) does not affect the results.

b0xtch commented 1 year ago

Could you elaborate on this?

kckeiks commented 1 year ago

Could you elaborate on this?

👍 I updated the title and added a description.

b0xtch commented 1 year ago

Can we create a bench for each crate we want to benchmark? And then just add one command to ci to run all the benches

kckeiks commented 1 year ago

I don't have any particular preference atm. I imagine that whoever picks this work can assess. I think the minimum requirement should be that the whole flow (querying indexer, requesting from node) is tested.

b0xtch commented 1 year ago

I don't have any particular preference atm. I imagine that whoever picks this work can assess. I think the minimum requirement should be that the whole flow (querying indexer, requesting from node) is tested.

not about preference, whether we decide to do a full benchmark crate to do all of this or are we benchmarking a specific crate.