grafana / xk6-browser

k6 extension that adds support for browser automation and end-to-end web testing via the Chrome Devtools Protocol
https://grafana.com/docs/k6/latest/javascript-api/k6-experimental/browser/
GNU Affero General Public License v3.0
327 stars 41 forks source link

Importing k6-browser results in K6_BROWSER_LOG affecting non-k6-browser console logging #1131

Open tom-miseur opened 7 months ago

tom-miseur commented 7 months ago

Brief summary

As soon as you add the import statement for k6/experimental/browser, console.x statements in the script are suppressed along with those emitted by k6-browser when setting K6_BROWSER_LOG (I'm assuming that this is unintended, given the name of the environment variable).

xk6-browser version

v0.47.0

OS

Windows 10

Chrome version

N/A

Docker version and image (if applicable)

No response

Steps to reproduce the problem

Set K6_BROWSER_LOG=error and run this script:

export const options = {
  scenarios: {
    checkout: {
      executor: 'shared-iterations',
      vus: 1,
      iterations: 1,
      options: {
        browser: {
          type: 'chromium',
        },
      },
    }
  },
}

export default function () {
  console.log('Hello, World!')
}
// prints `Hello, World!`

Repeat with the import added:

import { browser } from 'k6/experimental/browser'

export const options = {
  scenarios: {
    checkout: {
      executor: 'shared-iterations',
      vus: 1,
      iterations: 1,
      options: {
        browser: {
          type: 'chromium',
        },
      },
    }
  },
}

export default function () {
  console.log('Hello, World!')
}
// does not print `Hello, World!`

Expected behaviour

Only log entries emitted by k6-browser are affected (e.g. console-api messages).

Actual behaviour

All console.x output is affected.

strowk commented 1 week ago

I'd like to note that logs that are often relevant for fixing issues within tests, the ones coming from k6-browser itself (like. f.e "strict mode violation, multiple elements returned for selector query") are also affected by this and it is impossible to suppress console.error from erroneously written frontend code, while not suppressing logs from those relevant to test errors.