artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
7.94k stars 507 forks source link

Incorrect documentation and examples about HTTP's function and beforeScenario/afterScenario hooks #3336

Open peterhalicky opened 2 weeks ago

peterhalicky commented 2 weeks ago

Documentation about function and beforeScenario/afterScenario hooks of the HTTP engine states that the hooks receive two parameters - context and events. Also some examples only specify those two.

I've just spent an hour or so scratching my head why aren't my tests working when I just added an empty beforeScenario hook function in my test, until I found another example where those functions receive the 3rd argument next and I need to call it to let the scenario continue.

beforeRequest and afterResponse doc seems to have the same problem.

hassy commented 2 weeks ago

thanks for taking the time to open an issue @peterhalicky! We need to make it clearer in the docs that functions have to be async. It's tucked away elsewhere in the docs at the moment and is not easy to find.

peterhalicky commented 2 weeks ago

OK, looks like it's more complicated than I initially thought. Documentation says that hooks can be async. Installing @types/artillery brings in the type definitions, but those say that the hooks are supposed to return void -- but async functions will return Promise<void> which isn't the same thing, so you get a warning in the IDE and also artillery (nodejs) complains ("DeprecationWarning: Calling promisify on a function that returns a Promise is likely a mistake.").

In fact, as I just found, if a function is async, the next() call isn't needed -- but I still get those deprecation warnings above.