grafana / xk6-browser

The browser module adds support for browser automation and end-to-end web testing via the Chrome Devtools Protocol to k6.
https://grafana.com/docs/k6/latest/javascript-api/k6-browser/
GNU Affero General Public License v3.0
343 stars 41 forks source link

Early `Context` cancelation issue #1468

Open ankur22 opened 1 month ago

ankur22 commented 1 month ago

What?

In CI we're getting a lot of test failures which requires a rerun of the CI job for them to pass. We usually see something like this in the output:

--- FAIL: TestLifecycleGoto (0.00s)
    --- FAIL: TestLifecycleGoto/load (0.39s)
        lifecycle_wait_test.go:582: testBrowser: canceled

Example CI run: https://github.com/grafana/xk6-browser/actions/runs/11222847117/job/31196156631?pr=1456

Why?

This issue came about after we started working with the vu context to control the iteration's lifecycle.

How?

We need to take a look at the tests that fail and fix why the context is closing earlier than expected.

Tasks

### Tasks
- [ ] PR
- [ ] Update the k6 release notes and release tasks

Related PR(s)/Issue(s)

No response

inancgumus commented 1 week ago

This happens to me constantly in local runs as well. I've changed the issue's title to reflect that.

Interestingly, it happens when I run a suite of tests, not when I run these tests individually.

This is how I run the tests these days :(

go test ./browser ./common ./chromium ./tests \
  -skip="TestPageTargetBlank|TestKeyboardPress|TestPageOn|TestBrowserContextCookies|TestPageSetExtraHTTPHeaders" \
  -failfast -race

Another downside is that the integration test run takes 4X more time than before:

ok      github.com/grafana/xk6-browser/browser  3.363s
ok      github.com/grafana/xk6-browser/common   2.329s
ok      github.com/grafana/xk6-browser/chromium 2.268s
ok      github.com/grafana/xk6-browser/tests    96.516s