grafana / xk6-browser

The browser module adds support for browser automation and end-to-end web testing via the Chrome Devtools Protocol to k6.
https://grafana.com/docs/k6/latest/javascript-api/k6-browser/
GNU Affero General Public License v3.0
343 stars 41 forks source link

Nil pointer panic and wrong argument serialization in `waitForFunction()` #291

Closed mtarancon closed 2 years ago

mtarancon commented 2 years ago

Hi, I am trying to make the xk6-browser work for an application that has WebGL unity application, unfortunately I cant share the webGL because its still a WIP bur I want to test it with your amazing tool, I am having this issue:

ERRO[0001] panic: runtime error: invalid memory address or nil pointer dereference
goroutine 75 [running]:
runtime/debug.Stack()
    runtime/debug/stack.go:24 +0x65
go.k6.io/k6/js.(*VU).runFn.func1()
    go.k6.io/k6@v0.36.0/js/runner.go:782 +0x12b
panic({0x1da8bc0, 0x2ce1bb0})
    runtime/panic.go:838 +0x207
github.com/dop251/goja.AssertFunction.func1.1()
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/runtime.go:2273 +0x6e
panic({0x1da8bc0, 0x2ce1bb0})
    runtime/panic.go:838 +0x207
github.com/dop251/goja.(*vm).try.func1()
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/vm.go:539 +0x635
panic({0x1da8bc0, 0x2ce1bb0})
    runtime/panic.go:838 +0x207
github.com/grafana/xk6-browser/common.convertArgument({0x1df8880?, 0xc002f8a3c0?}, 0xc001c26f00, {0x0, 0x0})
    github.com/grafana/xk6-browser@v0.2.0/common/helpers.go:56 +0x41
github.com/grafana/xk6-browser/common.(*ExecutionContext).eval(0xc001c26f00, {0x220efe0, 0xc002dd0d20}, {0x60?, 0x80?}, {0x22172d0, 0xc002f8a020?}, {0xc002f88660, 0x4, 0x6})
    github.com/grafana/xk6-browser@v0.2.0/common/execution_context.go:229 +0x137f
github.com/grafana/xk6-browser/common.(*Frame).waitForFunction(0xc00067a000, {0x220efe0, 0xc002dd0d20}, {0x1f49170, 0x7}, {0x2216ea8?, 0xc0031dd020?}, 0x203000?, 0x203000?, 0x6fc23ac00, ...)
    github.com/grafana/xk6-browser@v0.2.0/common/frame.go:539 +0x68d
github.com/grafana/xk6-browser/common.(*Frame).WaitForFunction(0xc00067a000, {0x2216ea8, 0xc0031dd020}, {0x0, 0x0}, {0xc002d78250, 0x1, 0x1})
    github.com/grafana/xk6-browser@v0.2.0/common/frame.go:1394 +0x30c
github.com/grafana/xk6-browser/common.(*Page).WaitForFunction(0xc002f86000, {0x2216ea8, 0xc0031dd020}, {0x0, 0x0}, {0x0, 0x0})
    github.com/grafana/xk6-browser@v0.2.0/common/page.go:867 +0x13a
reflect.Value.call({0x1d941a0?, 0xc0007e1040?, 0x30c0f18?}, {0x1f445fd, 0x4}, {0xc002d646e0, 0x3, 0xc002d79178?})
    reflect/value.go:556 +0x845
reflect.Value.Call({0x1d941a0?, 0xc0007e1040?, 0xc0031dd020?}, {0xc002d646e0, 0x3, 0x3})
    reflect/value.go:339 +0xbf
github.com/dop251/goja.(*Runtime).wrapReflectFunc.func1({{0x2216ea8, 0xc00066d620}, {0xc002dc4080, 0x1, 0x6}})
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/runtime.go:1867 +0x259
github.com/dop251/goja.(*vm)._nativeCall(0xc001562700, 0xc002dafea0, 0x1)
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/vm.go:2725 +0x217
github.com/dop251/goja.call.exec(0xf?, 0xc001562700)
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/vm.go:2697 +0x95b
github.com/dop251/goja.(*vm).run(0xc001562700)
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/vm.go:410 +0x8b
github.com/dop251/goja.(*baseJsFuncObject)._call(0xc002d6e0b0, {{0x22176f8, 0x2d31b80}, {0xc002dab4e0, 0x1, 0x1}}, {0x0?, 0x0}, {0x22176f8, 0x2d31b80})
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/func.go:193 +0x3ff
github.com/dop251/goja.(*baseJsFuncObject).call(...)
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/func.go:203
github.com/dop251/goja.(*baseJsFuncObject).Call(0xc002d94820?, {{0x22176f8, 0x2d31b80}, {0xc002dab4e0, 0x1, 0x1}})
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/func.go:156 +0xc7
github.com/dop251/goja.AssertFunction.func1.2()
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/runtime.go:2278 +0x74
github.com/dop251/goja.(*vm).try(0x30?, 0x10630f6?)
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/vm.go:547 +0x176
github.com/dop251/goja.AssertFunction.func1({0x22176f8?, 0x2d31b80?}, {0xc002dab4e0?, 0x2af9480?, 0x203001?})
    github.com/dop251/goja@v0.0.0-20220110113543-261677941f3c/runtime.go:2277 +0x10c
go.k6.io/k6/js.(*VU).runFn(0xc002d57a40, {0x220efe0, 0xc002dbec60}, 0x40?, 0xc002384318, {0xc002dab4e0, 0x1, 0x1})
    go.k6.io/k6@v0.36.0/js/runner.go:787 +0x28b
go.k6.io/k6/js.(*ActiveVU).RunOnce(0xc0007e02c0)
    go.k6.io/k6@v0.36.0/js/runner.go:729 +0x395
go.k6.io/k6/lib/executor.getIterationRunner.func1({0x220efe0, 0xc002dbe9c0}, {0x2205020?, 0xc0007e02c0?})
    go.k6.io/k6@v0.36.0/lib/executor/helpers.go:145 +0x64
go.k6.io/k6/lib/executor.PerVUIterations.Run.func3({0x220ae48, 0xc002d57a40})
    go.k6.io/k6@v0.36.0/lib/executor/per_vu_iterations.go:239 +0x40a
created by go.k6.io/k6/lib/executor.PerVUIterations.Run
    go.k6.io/k6@v0.36.0/lib/executor/per_vu_iterations.go:252 +0xbba

Goja stack:
native 
ERRO[0001] a panic occurred in VU code but was caught: runtime error: invalid memory address or nil pointer dereference  executor=per-vu-iterations scenario=default
WARN[0001] url:http://localhost:3092/packs/media/images/logos/instagram-f7b3dff34fba7bf43d83022425cb2ef3.png method:GET err:error fetching response body: context canceled  category="Response:bodySize:fetchBody" elapsed="0 ms" goroutine=184

running (00m01.1s), 0/1 VUs, 1 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs  00m01.1s/10m0s  1/1 iters, 1 per VU
ERRO[0001] err:websocket: close 1006 (abnormal closure): unexpected EOF  category="Connection:handleIOError" elapsed="0 ms" goroutine=41

     browser_dom_content_loaded.......: avg=19.4ms   min=30µs    med=36µs     max=58.13ms  p(90)=46.51ms  p(95)=52.32ms 
     browser_first_contentful_paint...: avg=74.74ms  min=74.74ms med=74.74ms  max=74.74ms  p(90)=74.74ms  p(95)=74.74ms 
     browser_first_paint..............: avg=74.74ms  min=74.74ms med=74.74ms  max=74.74ms  p(90)=74.74ms  p(95)=74.74ms 
     browser_loaded...................: avg=167.99µs min=154µs   med=167.99µs max=182µs    p(90)=179.2µs  p(95)=180.6µs 
     data_received....................: 366 kB 340 kB/s
     data_sent........................: 8.7 kB 8.1 kB/s
     http_req_connecting..............: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_duration................: avg=113.63ms min=6.54ms  med=21.28ms  max=630.16ms p(90)=285.16ms p(95)=457.66ms
     http_req_receiving...............: avg=94.14ms  min=2ms     med=6ms      max=627ms    p(90)=256.2ms  p(95)=441.59ms
     http_req_sending.................: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_tls_handshaking.........: avg=0s       min=0s      med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_reqs........................: 7      6.513282/s
     vus..............................: 1      min=1      max=1
     vus_max..........................: 1      min=1      max=1

Im using MacOS Catalina 10.15.7

 ./xk6-browser version
k6 v0.36.0 ((devel), go1.18, darwin/amd64)
imiric commented 2 years ago

Thanks for opening this issue @mtarancon!

I took a look at it and it's not a problem with WebGL or any site for that matter, but with our waitForFunction() implementation. Here's a minimal script that reproduces the panic:

import launcher from 'k6/x/browser';

export default function() {
  const browser = launcher.launch('chromium', {
    headless: true,
  });
  const context = browser.newContext();
  const page = context.newPage();

  page.waitForFunction(() => true);

  page.close();
  browser.close();
}

This happens on current main (990a95c).

There are several issues here though... :disappointed: Some notes for developers:

Given that the first problem masks the second one, let's use this issue for tracking both bugs.

Thanks again for reporting this @mtarancon. We'll try to get this fixed in the upcoming v0.3.0 release (end of this month). :crossed_fingers:

In the meantime, could you write the test without using waitForFunction()? Maybe use waitForSelector() and then evaluate() if you need to interact with the WebGL object?

The good news is that this is not related to WebGL :tada: I tested with this demo, and it worked without issues on my machine (ran at 60FPS).

imiric commented 2 years ago

Hi @mtarancon, the issue you were running into should be fixed on the current main branch. You can build and use your own binary right now to test it, or wait until we make a release sometime at the end of this month or beginning of May.

To reiterate, this wasn't an issue with WebGL and you shouldn't have this specific problem anymore, but like I mentioned on the forum, there are still many unknowns with testing WebGL sites, and running such tests in k6 Cloud will remain unsupported for some time.

If you have other blocking issues please create a new issue here, or ask questions on the forum. Thanks!