artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
7.95k stars 507 forks source link

"histogram" event is not being triggered when using an event listener in plugin #2558

Closed servohatred closed 6 months ago

servohatred commented 6 months ago

Version info:

2.0.7

Running this command:

 artillery run mytestfile.yaml

I expected to see this happen:

The histogram event to be fired and my callback to be executed

Instead, this happened:

the event was not fired

In my test processor(playwright engine) I've been using events.emit("histogram", "customevent", duration) to record my custom metrics, now I want to grab my test summary to fetch my event results programatically so I can send this data to a reporter. I am trying that by doing so(also got this from playwright reporter code in the publish metrics plugin): events.on('histogram', (name, value, metadata) => {console.log("this event was fired")} ) this event never gets triggered even if it's displayed in the console reporter summary. Also tried by using a similar approach I found in the publish metrics datadog plugin: events.on('stats', (stats) => {console.log(stats.reporter())}) this event works and it's fired but all the reports I get from this approach are empty and I dont get any data related to my custom metrics(custom metrics, summaries and counters are always empty) .

Is there a way to collect the data from the test summary in the console output? I've been reading a report json file but due to concurrency issues with the files I need to find a way to send my data from an after hook or a plugin cleanup

hassy commented 6 months ago

hi @servohatred 👋 emitting metrics with histogram (and counter) is one-way - all emitted metrics are consumed and then aggregated by Artillery's metrics engine, and then aggregated reports are emitted via stats and done events to consumers (i.e. plugins).

Not sure why you're not receiving the stats event though - it's built in functionality that e.g. the publish-metrics plugin relies on, and should be working.