grafana / xk6-browser

k6 extension that adds support for browser automation and end-to-end web testing via the Chrome Devtools Protocol
https://grafana.com/docs/k6/latest/javascript-api/k6-experimental/browser/
GNU Affero General Public License v3.0
328 stars 42 forks source link

Clicking on a link that opens a new tab never navigates to the href link #827

Closed ankur22 closed 3 weeks ago

ankur22 commented 1 year ago

Brief summary

When a link that opens in a new tab is clicked on, the navigation never completes in the new tab and seems to hang on the about:blank page.

xk6-browser version

v0.43.1

OS

Version 13.2 (22D49)

Chrome version

Version 110.0.5481.177 (Official Build) (arm64)

Docker version and image (if applicable)

NA

Steps to reproduce the problem

This is the script that I'm trying to test, which should navigate to the reviews section in a new tab when the link has been clicked on:

import { browser } from "k6/x/browser";

export const options = {
  scenarios: {
    ui: {
      executor: "shared-iterations",
      options: {
        browser: {
          type: "chromium",
        },
      },
    },
  }
};

export default async function () {
  const page = browser.newPage();

  try {
    await page.goto("https://k6.io", { waitUntil: "networkidle" });

    await Promise.all([
      page.waitForNavigation(),
      page.locator('a[href="https://www.g2.com/products/k6/reviews"]').click(),
    ]);
  } finally {
    page.close();
  }
}

In the new tab it never navigates to the link.

Screenshot 2023-03-14 at 11 23 18

The output of the test is:

ERRO[0030] communicating with browser: websocket: close 1006 (abnormal closure): unexpected EOF  category=cdp elapsed="0 ms" goroutine=51
panic: GoError: creating a new page: initializing networking *network.EnableParams: websocket: close 1006 (abnormal closure): unexpected EOF
running (00m30.0s), 1/1 VUs, 0 complete and 0 interrupted iterations
goroutine 54 [running]:--------------------------] 1 VUs  00m30.0s/10m0s  0/1 shared iters
go.k6.io/k6/js/common.Throw(...)
        go.k6.io/k6@v0.42.1-0.20230130080633-582ec4d3940c/js/common/util.go:20
github.com/grafana/xk6-browser/k6ext.Panic.func1(0x0?, {0x14000abcd70?, 0x14002464b54?, 0x140006719c0?})
        github.com/grafana/xk6-browser@v0.0.0-00010101000000-000000000000/k6ext/panic.go:35 +0x7c
github.com/grafana/xk6-browser/k6ext.sharedPanic({0x1055492e0, 0x1400099d040}, 0x14000671af0, {0x14000abcd70?, 0x1, 0x1})
        github.com/grafana/xk6-browser@v0.0.0-00010101000000-000000000000/k6ext/panic.go:76 +0x274
github.com/grafana/xk6-browser/k6ext.Panic({0x1055492e0?, 0x1400099d040?}, {0x104f29892?, 0x140014fb710?}, {0x14000abcd70?, 0x14001cdffe0?, 0x20?})
        github.com/grafana/xk6-browser@v0.0.0-00010101000000-000000000000/k6ext/panic.go:37 +0x58
github.com/grafana/xk6-browser/common.(*Browser).onAttachedToTarget(0x14000f1bd40, 0x140023126a0)
        github.com/grafana/xk6-browser@v0.0.0-00010101000000-000000000000/common/browser.go:294 +0x73c
github.com/grafana/xk6-browser/common.(*Browser).initEvents.func1()
        github.com/grafana/xk6-browser@v0.0.0-00010101000000-000000000000/common/browser.go:180 +0x260
created by github.com/grafana/xk6-browser/common.(*Browser).initEvents
        github.com/grafana/xk6-browser@v0.0.0-00010101000000-000000000000/common/browser.go:165 +0x194

Expected behaviour

The navigation to the review page in the new tab completes.

Actual behaviour

The navigation to the review page in the new tab never completes.

### Tasks
- [ ] https://github.com/grafana/xk6-browser/pull/1334
- [ ] Release notes
inancgumus commented 1 year ago

@ankur22, confirmed 👍

dineshtcs commented 12 months ago

This is a critical functionality required in any browser based automation test. May I know if this bug has been fixed please ?

sumitbhowmick commented 3 months ago

This issue is still present with latest k6 version 50. As a result we are not able to use k6 browser module.

ankur22 commented 2 months ago

I've taken a look at this issue. Unfortunately there isn't a solution and I'm still not sure what k6 browser is or isn't doing that is causing the issue in the first place. We're no closer to solving the issue.

When a test or a user action to click on a link is performed on a link that opens a new browser window, the CDP msgs from chrome instructs k6 browser that a new page is to be created. k6 browser attempts to perform the necessary tasks required to set one up. The problem is that some required CDP requests to create a new page and set it up for testing do not get a response back from Chrome.

Another issue worth pointing out is that the new window looks to be loading something but it is unclear exactly what it is trying to load.

I've compared CDP request/response data against Playwright to see if there are any clues as to what the differences are, and tried to setup chrome with the same flags, but nothing so far has helped resolve this incorrect behaviour when a new window opens after clicking on a link.

I've attached the two cdp request response files for the same test but one from k6 and the other from PW. Some changes were done in k6 browser to try to make it behave the same as Playwright including, but non of these helped:

The test itself is:

import { browser } from "k6/x/browser";

export const options = {
  scenarios: {
    ui: {
      executor: "shared-iterations",
      options: {
        browser: {
          type: "chromium",
        },
      },
    },
  }
};

export default async function () {
  const page = browser.newPage();

  await page.goto("https://k6.io/", { waitUntil: "networkidle" });

  await Promise.all([
    page.waitForNavigation(),
    page.locator('a[href="https://www.thoughtworks.com/radar/tools/k6"]').click(),
  ]);

  page.close();
}

NOTE: The page.waitForNavigation doesn't make sense in this case since the navigation occurs on a different page, but it helps highlight the issue due to the timeout, although maybe it's better to assert that the correct navigation occurs and a specific element is on the page. So far though I've ran all tests in headful mode and just observing the behaviour in real time.

The CDP requests in the k6 logs that never get a response are:

-> {"id":55,"sessionId":"C32A24D7DD551C4612E54916B76BFD46","method":"Emulation.setDeviceMetricsOverride","params":{"width":1280,"height":799,"deviceScaleFactor":1,"mobile":false,"screenWidth":1280,"screenHeight":720,"screenOrientation":{"type":"landscapePrimary","angle":90}}}
-> {"id":56,"sessionId":"C32A24D7DD551C4612E54916B76BFD46","method":"Page.getFrameTree"}
-> {"id":57,"sessionId":"C32A24D7DD551C4612E54916B76BFD46","method":"Page.enable"}
-> {"id":58,"sessionId":"C32A24D7DD551C4612E54916B76BFD46","method":"Network.enable","params":{}}
-> {"id":59,"sessionId":"C32A24D7DD551C4612E54916B76BFD46","method":"Page.setLifecycleEventsEnabled","params":{"enabled":true}}
-> {"id":60,"sessionId":"C32A24D7DD551C4612E54916B76BFD46","method":"Log.enable"}

In the PW logs all CDP requests receive a response.

newtab2-cdp-k6.log newtab2-cdp-pw.log

EDIT: When i manually reload the new window, things are unblocked. Chrome first sends something like:

<- {"method":"Page.frameStartedLoading","params":{"frameId":"4483871EC8055A41A4EC7E518A38E443"},"sessionId":"B15A62A49F84470481111EE45AEEDF03"}"

and then we start to see the responses from the requests that were made earlier.

Fran-VLC commented 2 months ago

We are really interested on fixing it too. It limit us because there are cases that we are not able to cover with k6.

inancgumus commented 2 months ago

@ankur22 This works (click on the element with the Command key (on macOS)):

export default async function () {
  const page = browser.newPage();

  await page.goto("https://k6.io/", { waitUntil: "networkidle" });

  page.evaluate(() => {
    function simulateCommandClick(element) {
      var event = new MouseEvent("click", {
        view: window,
        bubbles: true,
        cancelable: true,
        ctrlKey: true, // On Windows/Linux Ctrl+Click would be similar to Command+Click on Mac
        metaKey: true, // On Mac, metaKey represents the Command key
      });

      // Dispatch the event on the provided element
      element.dispatchEvent(event);
    }
    const el = document.querySelector('a[href="https://www.thoughtworks.com/radar/tools/k6"]');
    simulateCommandClick(el);
  });

  sleep(60);

  page.close();
}

Comparing the log output of this one with the usual k6-browser Click can give us a clue.

ankur22 commented 2 months ago

We have found the correct solution. We need to priortise the CDP request RunIfWaitingForDebugger, which will:

  1. Unblock the new window;
  2. Allow the setup of the new window;
  3. Navigate to the link.

A fix should be available in the next release (v0.52.0)