search

PragmaticFlow / NBomber

Modern and flexible load testing framework for Pull and Push scenarios, designed to test any system regardless a protocol (HTTP/WebSockets/AMQP etc) or a semantic model (Pull/Push).
https://nbomber.com
Other
2.07k stars 131 forks source link

Add threshold for statistics #355

Open AntyaDev opened 3 years ago

AntyaDev commented 3 years ago

https://k6.io/docs/using-k6/thresholds/

pavlogrushetsky commented 2 years ago

Hey @AntyaDev, do you mean here something like the following (conceptually), or something more sophisticated?

image

AntyaDev commented 2 years ago

Wow, looks really sexy!!!

AntyaDev commented 2 years ago

I think these thresholds should be applied after the test is finished. NodeStats |> applyTresholds [myTresholds]

AntyaDev commented 2 years ago

Otherwise, it should be somehow integrated into the whole execution pipeline

pavlogrushetsky commented 2 years ago

Okay, great! Just to be clear, this is just a prototype of how I feel I'd like to be able to configure this as a user (not a real implementation in the pipeline) 😄 By the way, do you have any guidelines/recommendations/requirements for the contributors? I noticed that to be able to expose these new models, I need to make changes in NBomber.Contracts, and NBomber itself references NBomber.Contracts as a Nuget package. How should the changes be coordinated between these two projects? Do you have some sort of a prerelease concept for the NBomber.Contracts package that could be used in the scope of a feature development in the NBomber project? I hope, I'm not missing something 🤞 but I would appreciate any guidance. I'd like to dig deeper into this if I have some spare time 😅

pavlogrushetsky commented 2 years ago

And this is a prototype of how this functionality could look like in C#. Please, let me know if this seems good to you.

image

pavlogrushetsky commented 2 years ago

Hey @AntyaDev, here is a draft PR with a rough implementation of the core thresholds logic - https://github.com/PragmaticFlow/NBomber/pull/465. It is rather for review and conversation, because the work is still in progress, and there are some other things I may need to implement/fix. But I would appreciate if you could take a look at it.

AntyaDev commented 2 years ago

Ok, looks amazing. I will take a look and comment.