grafana / xk6-browser

k6 extension that adds support for browser automation and end-to-end web testing via the Chrome Devtools Protocol
https://grafana.com/docs/k6/latest/javascript-api/k6-experimental/browser/
GNU Affero General Public License v3.0
337 stars 42 forks source link

Panic on attaching iframe when getting browser window ID #1224

Closed ankur22 closed 2 weeks ago

ankur22 commented 6 months ago

Brief summary

Panic on attaching iframe when getting browser window ID.

panic: GoError: attaching iframe: attaching iframe target ID 153A0D279584840F49FB1E718A8297AB to session ID DA84F3DFA9B0CA220314557F25BC8B37: getting browser window ID: No web contents for the given target id (-32000)

goroutine 97 [running]:
go.k6.io/k6/js/common.Throw(...)
    go.k6.io/k6/js/common/util.go:20
github.com/grafana/xk6-browser/k6ext.Panic.func1(0x155d220?, {0xc0018611c0?, 0x15fa301?, 0xc001bae5c0?})
    github.com/grafana/xk6-browser@v1.3.0/k6ext/panic.go:35 +0x74
github.com/grafana/xk6-browser/k6ext.sharedPanic({0x1b58098, 0xc000e131d0}, 0xc0026bdc80, {0xc0018611c0, 0x2, 0x2})
    github.com/grafana/xk6-browser@v1.3.0/k6ext/panic.go:64 +0x254
github.com/grafana/xk6-browser/k6ext.Panic({0x1b58098?, 0xc000e131d0?}, {0x17adf8a?, 0x294f5d0?}, {0xc0018611c0?, 0x0?, 0xc0026bde10?})
    github.com/grafana/xk6-browser@v1.3.0/k6ext/panic.go:37 +0x5a
github.com/grafana/xk6-browser/common.(*FrameSession).onAttachedToTarget(0xc0007d8540, 0xc001861000)
    github.com/grafana/xk6-browser@v1.3.0/common/frame_session.go:953 +0x671
github.com/grafana/xk6-browser/common.(*FrameSession).initEvents.func1()
    github.com/grafana/xk6-browser@v1.3.0/common/frame_session.go:282 +0x368
created by github.com/grafana/xk6-browser/common.(*FrameSession).initEvents in goroutine 73
    github.com/grafana/xk6-browser@v1.3.0/common/frame_session.go:223 +0x19b

xk6-browser version

v1.3.0 (browser)

OS

NA

Chrome version

NA

Docker version and image (if applicable)

NA

Steps to reproduce the problem

Test runs: 2398747, 2398780, 2688013, 2688024, 2688076, 2688087, 2842218, 2842268, 2842315

Expected behaviour

Not to Panic.

Actual behaviour

Panic.

### Tasks
- [ ] https://github.com/grafana/xk6-browser/pull/1406
- [ ] https://github.com/grafana/k6/pull/3905
matthisholleville commented 1 month ago

Hello ! Any news or Workaround for this issue ?

cc. @inancgumus

matthisholleville commented 1 month ago

To provide more context, we have been almost continuously affected by this issue since 07/18. We're not yet 100% sure, but it seems to stem from an upgrade of our targets from Angular version 16 to 17. We will attempt a rollback and keep you updated. We run all of our scenarios on k8s.

We are using the latest version of k6 available (0.52.0) with Chromium version 119.0.6045.159.

We can reproduce the issue locally when running our scenarios through a Docker image. However, when running the scenarios on our local machines (Mac M1 and Chromium), there are no issues.

In the dashboard below, we observe significant instability in our scenarios.

Without a solution or workaround, we might have to resort to using a different tool. 😢

image
inancgumus commented 1 month ago

Hi @matthisholleville. This issue was reported on the community forum before here. We made some improvements. However, this issue currently exists. Can you send us the Dockerfile, scripts, and sample scripts so that we can reproduce this issue locally? This will allow us to come up with a solution faster. Thanks!

matthisholleville commented 1 month ago

Hi @inancgumus ! Thank you.

We have identified the cause. It is due to an iframe injected by a third-party library (Stripe). Disabling this iframe in our non-production environments resolved the issue.

image

Here is the test context that you can reuse:

// scenario
await page.goto("https://app.agicap.com/fr/app/partners/cdn/connect", { waitUntil: 'load' });
await page.waitForSelector(`[data-test="${partnerSelector}"]`, { state: "visible", timeout: 20000 });
FROM grafana/xk6:0.12.1 as builder
WORKDIR /build
RUN xk6 build v0.52.0 --output "/tmp/k6" --with github.com/acuenca-facephi/xk6-read@v1.0.0-rc1

FROM grafana/k6:0.52.0-with-browser@sha256:d054d029e1518ce2bff24b2dc9a8d57270816c8bcec5b2235e2328ae6368e20a
COPY --from=builder /tmp/k6 /usr/bin/k6

ENV K6_BROWSER_HEADLESS=true
ENV K6_AGICAP_ACCOUNT_WEBSITE_URL=https://app.agicap.com

WORKDIR /home/k6
COPY . .
WORKDIR /home/k6/scenarios
ENTRYPOINT ["k6"]

You need to run the test multiple times to see an error. It is quite flaky, but I can reproduce it frequently.

inancgumus commented 1 month ago

Thanks, @matthisholleville, this is helpful 🙇 We'll consider this information while working on this issue 👍

matthisholleville commented 1 month ago

@inancgumus We will probably change the way we load this iframe. In case you want to reproduce it from a local page.

The iframe is loaded from this:

<script src="https://js.stripe.com/v3/"></script>
matthisholleville commented 1 month ago

@inancgumus We have just removed the loading of this iframe when it is not necessary, and we observe a SIGNIFICANT improvement across all our scenarios.

It would be interesting to understand why this particular iframe is causing issues.

scenarios-update

amirmuminovic commented 1 month ago

Hi @matthisholleville , I am facing a similar issue, can you share more info about how you removed/disabled the iframe?

matthisholleville commented 1 month ago

Hi @amirmuminovic ! First of all, in my opinion the problem is not with the iframe but with k6. We use other tools for our smokes ( pingdom ) and we've never seen this error.

For our part, this iframe was loaded in the DOM of all our pages. However, it was only useful in a specific part of the application. We only load it when we try to access the specific part.

It is currently IMPOSSIBLE for us to create a k6 scenario on the part that currently contains this iframe

ankur22 commented 2 weeks ago

@matthisholleville and @amirmuminovic we believe we have fixed this issue and it has been merged into main. This will make the next release of k6 (v0.54.0).

In the meantime if you wish to try it out with the fix and are comfortable with the Go ecosystem you can do so by using xk6:

  1. Download the xk6 binary with: go install go.k6.io/xk6/cmd/xk6@latest
  2. Build a new k6 binary with: xk6 build --with github.com/grafana/xk6-browser@main

Thank you for the help in resolving this issue!

matthisholleville commented 2 weeks ago

Thank you @ankur22 & @inancgumus !