search

grafana / xk6-websockets

GNU Affero General Public License v3.0
17 stars 6 forks source link

Investigate more autobahn failures and improvement to the test #4

Open mstoykov opened 2 years ago

mstoykov commented 2 years ago

There are still some test (from the very late ones) that are failing. Unfortunately it isn't immediately obvious what they test or what is failing exactly so that will likely take quite a lot of work on it's own :(.

It might also be due to the websocket library we are using which is still looking for maintainers and isn't all that up to date.

We likely will want to at least try other variants in the future as well.

imiric commented 1 year ago

Hey @mstoykov, I'm not clear on what needs to be done here.

I see that gorilla/websocket mentions Autobahn tests in their README, but I'm confused why we need to run the suite, since we're not implementing the WS protocol ourselves, and both k6/ws and k6/experimental/websockets use gorilla/websocket.

So why (and how) should we run this? And what can/should we do in case of any failures?

mstoykov commented 1 year ago

The autobahn test was added by @codebien in https://github.com/grafana/xk6-websockets/pull/1 and can be run as explained in the README.

It already helped with some fixes as shown in https://github.com/grafana/xk6-websockets/pull/3/files.

I have no idea though what the current failures are due to (you will need to run them) as I find the documentation around it to be abysmal. It is likely they are to do with less relevant features or expectations, given that much later tests fail.

mstoykov commented 1 year ago

It turns out that the reports folder under the autobahn do get populated with a really nice html report that can be looked at in a browser which has a ton of information :tada:.

Obviously I didn't notice this the previous time I looked at it :facepalm: