stress test

[hwchen]

stress test acs cubes

[hwchen]

from @jspeis : could be useful to monitor which cubes might stress the server the most (e.g. do we need to hide ygsar poverty cubes from UI)

[hwchen]

from slack:

Something that would be useful is if we could also run some tests on loading 5, 10, 15, 20, 25, 30 profile pages at once hwchen [4:38 PM] This is all cold start, right? jspeiser [4:38 PM] well after internal site cache but otherwise cold for reference ... when loading 6 simultaneous profile pages at once from cold cache the current data USA starts to go slow but we'll just want to see where things break and if we get any 500 errors along the way hwchen [4:39 PM] ok. How did you do simultaneous profile loads? And did you pick the profile pages randomly? jspeiser [4:39 PM] and use geo profiles because each geo profile will in turn execute probably 100 API requests hwchen [4:39 PM] ah, geo profiles. That makes sense. jspeiser [4:40 PM] randomly from among states/places with >=100k population so that they are larger api calls so the thruput is actually something like 100x the number of geo profiles that we can load and if we do run into errors, if you can dig into where the bottlenecks are? (e.g. is it in mondrian? in monet? in Dave's node logiclayer?) hwchen [4:41 PM] yup, makes sense last thing... how did you load the profile pages simultaneously? jspeiser [4:42 PM] In the past I've used a tool called ab but there's also this: https://github.com/giltene/wrk2 GitHubGitHub giltene/wrk2 A constant throughput, correct latency recording variant of wrk - giltene/wrk2 hwchen [4:42 PM] interesting, I've heard of wrk (but I usually see people use it for very basic throughput "hello world" framework tests) ok, I think I got it, thanks!

[hwchen]

Trying to get under 20 s.

try up to 10 simultaneous profiles.

profile api calls. (check exactly what the queries are in logic layer) profile raw api calls versus logic layer.

[jspeis]

another great way to understand performance bottlenecks will be to enable query performance logging, while trying to load an uncached profile page. see example usage here:

https://www.monetdb.org/Documentation/Cookbooks/SQLrecipes/QueryHistory

this can help identify query/tables that are the slowest to load and could benefit from optimization.

[hwchen]

Monetdb perf seems acceptable, stress-testing bugs are dealt with in other issues, so closing this.