search

grafana / xk6-browser

k6 extension that adds support for browser automation and end-to-end web testing via the Chrome Devtools Protocol
https://grafana.com/docs/k6/latest/javascript-api/k6-experimental/browser/
GNU Affero General Public License v3.0
325 stars 41 forks source link

Fix flaky tracing test that fails with web_vital span err #1150

Open inancgumus opened 5 months ago

inancgumus commented 5 months ago

Brief summary

When the integration tests are ran in CI, the workflow often fails on one of the TestTracing, specifically the web_vital sub test.

When this test fails we can usually rerun the failed workflow and the test will pass the second time round, so it's flakey. It would be good if this wasn't the case so that it gives us confidence in our tests and workflows, as well as making it easier to contribute to for external contributors.

xk6-browser version

fcc7f9e5f45b099d8516cb7cc38662b9110b3c23

OS

NA

Chrome version

NA

Docker version and image (if applicable)

NA

Steps to reproduce the problem

Run the test in CI, e.g.: https://github.com/grafana/xk6-browser/actions/runs/8138217177/job/22238484745?pr=1228

Expected behaviour

The integration test passes.

Actual behaviour

It fails with:

--- FAIL: TestTracing (3.71s)
    tracing_test.go:161: 
            Error Trace:    /home/runner/work/xk6-browser/xk6-browser/tests/tracing_test.go:161
            Error:          Received unexpected error:
                            "web_vital" span was not found
            Test:           TestTracing
### Tasks
- [ ] https://github.com/grafana/xk6-browser/pull/1262
- [ ] https://github.com/grafana/k6/pull/3709
inancgumus commented 4 months ago

It happens every other day. Here's the latest one:

time="2024-03-05T06:01:29Z" level=error msg="error on evaluating window.k6SpanId: evaluating JS in global context: context canceled" category="FrameSession:onFrameNavigated" elapsed="0 ms" source=browser
time="2024-03-05T06:01:29Z" level=error msg="Failed to load resource: the server responded with a status of 404 (Not Found)" browser_source=network line_number=0 source=browser stacktrace="<nil>" url="http://127.0.0.1:35093/static/empty.html"
time="2024-03-05T06:01:29Z" level=error msg="error on evaluating window.k6SpanId: evaluating JS in global context: context canceled" category="FrameSession:onFrameNavigated" elapsed="0 ms" source=browser
--- FAIL: TestTracing (6.88s)
    tracing_test.go:1[61](https://github.com/grafana/xk6-browser/actions/runs/8151736116/job/22280064609#step:4:62): 
            Error Trace:    /home/runner/work/xk6-browser/xk6-browser/tests/tracing_test.go:161
            Error:          Received unexpected error:
                            "web_vital" span was not found
            Test:           TestTracing

Interesting points are:

"Failed to load resource: the server responded with a status of 404 (Not Found)" -> FrameSession:onFrameNavigated -> "error on evaluating window.k6SpanId -> "evaluating JS in global context: context canceled"