grafana / xk6-browser

k6 extension that adds support for browser automation and end-to-end web testing via the Chrome Devtools Protocol
https://grafana.com/docs/k6/latest/javascript-api/k6-experimental/browser/
GNU Affero General Public License v3.0
337 stars 42 forks source link

Incorrect error message when docker image ran on ARM machine #1029

Open ankur22 opened 1 year ago

ankur22 commented 1 year ago

Brief summary

When a test script is ran against a containerised version of k6 and chrome on a ARM machine, it will result in the following error:

time="2023-09-04T12:48:08Z" level=error msg="process with PID 18 unexpectedly ended: signal: trace/breakpoint trap" category=browser elapsed="0 ms" source=browser
time="2023-09-04T12:48:08Z" level=error msg="Uncaught (in promise) GoError: browser not found in registry. make sure to set browser type option in scenario definition in order to use the browser module\n\tat github.com/grafana/xk6-browser/browser.mapBrowser.func6 (native)\n\tat file:///-:20:15(4)\n" executor=shared-iterations scenario=ui

When the same script is ran on a machine running on a AMD64 processor the test passes without an issue.

I believe the issue is that Chrome is crashing in the running container but we're masking the original issue with the wrong error message.

xk6-browser version

v0.46.0

OS

Mac on ARM

Chrome version

NA

Docker version and image (if applicable)

Latest

Steps to reproduce the problem

  1. Try to run any valid test script (such as fillform.js) on a MAC running on a ARM processor with docker run --rm --cap-add=SYS_ADMIN -i grafana/k6:0.46.0-with-browser run - <test.js

Expected behaviour

The test passes as expected.

Actual behaviour

The test fails with:

time="2023-09-04T12:48:08Z" level=error msg="process with PID 18 unexpectedly ended: signal: trace/breakpoint trap" category=browser elapsed="0 ms" source=browser
time="2023-09-04T12:48:08Z" level=error msg="Uncaught (in promise) GoError: browser not found in registry. make sure to set browser type option in scenario definition in order to use the browser module\n\tat github.com/grafana/xk6-browser/browser.mapBrowser.func6 (native)\n\tat file:///-:20:15(4)\n" executor=shared-iterations scenario=ui
ankur22 commented 1 year ago

I had a quick look. Unfortunately there's two issues:

  1. When the chrome process ends unexpectedly, there's currently no way to transfer this information to the rest of the system so that it knows about this problem.
  2. Even if we added a simple fix to store the error state of chrome ending in the browser instance, there's a chance that either the iterEnd or exit events would have fired which will clear the browser before the system can check the state of chrome.

I believe the browser is being set just before or as Chrome is ending. The process of chrome ending starts the termination of the whole test run, which in turn clears the browser away. When the test comes to retrieve the browser it's no longer there, and that is why we print the incorrect reason for the test terminating.