Performance test degradation

[ScharfViktor]

last 4 days we see some degradation in the nightly k6 testing. see here https://grafana.k6.infra.owncloud.works/

k6-test is red in CI because thresholds on metrics 'http_req_duration (rate<0.02), http_req_failed (p(95)<500ms)' have been crossed

example:

• https://drone.owncloud.com/owncloud/ocis/40918/58/1

last green test was 6 days ago https://drone.owncloud.com/owncloud/ocis/40699/58/1

how to test:

• create PR with [k6-test] in the title
• see CI output

[kobergj]

Since last successful run was on 6th November at 1am the degradation must have happened on Nov 6th. Luckily there were not so many relevant PRs that day. It must be one of these:

• https://github.com/owncloud/ocis/pull/10476
• ~https://github.com/owncloud/ocis/pull/10481~
• ~https://github.com/owncloud/ocis/pull/10494~

Other PRs on that day only touch docs or unrelated systems.

10476 adds 25 workers to the frontend service. Maybe that is somehow affecting request duration? Frontend service taking too much mem/cpu? THATS THE BADDY

10481 bumps a grpc dependency. That could impact grpc request duration. But since we bumped it, it should be quicker if anything, shouldn't it? Tested - No impact on performance

~#10494 bumps reva. But I don't see any changes in there that would affect performance. Just some cs3api bump.~

There is also the possibility that the infrastructure changed. Where there any changes to it last week?

[ScharfViktor]

There is also the possibility that the infrastructure changed. Where there any changes to it last week?

there have been no changes to the infrastructure in the last week.

[micbar]

Have there been some changes to the k6 tests?

[ScharfViktor]

Have there been some changes to the k6 tests?

Nope, there was nothing.