omegaup / prod

Deployment manifests for omegaUp
BSD 3-Clause "New" or "Revised" License
1 stars 3 forks source link

Investigate why there is a 100ms-200ms difference with current production #2

Closed lhchavez closed 2 years ago

lhchavez commented 2 years ago

It may be because we have one or two extra hops (loadbalancer -> nginx -> php vs nginx -> php), but ideally we should see if there's anything we can do about it.

lhchavez commented 2 years ago

A bit of optimizing later and we're now pretty close! most notably removing the IPv6 support from the ELB (since EKS still doesn't support IPv6, so all those requests were timing out, ha) did most of the trick:

$ echo "GET https://k8s.omegaup.com" | vegeta attack -duration=30s -rate=50 | vegeta report
Requests      [total, rate, throughput]         1500, 50.03, 49.86
Duration      [total, attack, wait]             30.087s, 29.98s, 107.251ms
Latencies     [min, mean, 50, 90, 95, 99, max]  99.053ms, 142.88ms, 105.223ms, 269.756ms, 359.827ms, 477.787ms, 528.652ms
Bytes In      [total, mean]                     22345500, 14897.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:1500
Error Set:
$ echo "GET https://omegaup.com" | vegeta attack -duration=30s -rate=50 | vegeta report
Requests      [total, rate, throughput]         1500, 50.03, 49.86
Duration      [total, attack, wait]             30.084s, 29.98s, 103.702ms
Latencies     [min, mean, 50, 90, 95, 99, max]  102.628ms, 114.437ms, 105.185ms, 110.462ms, 123.518ms, 403.771ms, 571.472ms
Bytes In      [total, mean]                     22345500, 14897.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:1500
Error Set: