apache / jmeter

Apache JMeter open-source load testing tool for analyzing and measuring the performance of a variety of services
https://jmeter.apache.org/
Apache License 2.0
8.42k stars 2.1k forks source link

Create a new TestElement that runs after PostProcessors and Assertions #6047

Open dgetzlaf opened 1 year ago

dgetzlaf commented 1 year ago

Use case

Hello there,

in our JMX we use a PostProcessor to persist some metrics, if an Sample failed. To be applied on each Sampler, we put it in the jmeterTestPlan. image

When an Assertion is added to a specific Sampler, and it failes, the PostProcessor does persist this error. The reason is, JMeter executes PostProcessors before Assertions.

Since the existing execution order in JMeter should not be touched (possible impacts for existing TestPlans), it would be great to have a new Object, that runs after all Assertions were evaluated.

Possible solution

Add a new PostProcessor that is executed in "the final stage" of the LiveCycle of a Sampler.

My proposal for a name would be "FinalPostProcessor".

Possible workarounds

No response

JMeter Version

5.6.2

Java Version

openjdk version "17.0.7" 2023-04-18 LTS

OS Version

Microsoft Windows [Version 10.0.19045.3155]

vlsi commented 1 year ago

Could you suggest use cases for such elements? Does JSR223 Assertion resolve the case?

dgetzlaf commented 1 year ago

@vlsi thanks for your quick response! :)

The JSR223 Assertion supports me a bit -> When a Sampler has a specific Assertion, I can add this Element for this Sampler. But since I long for a "more generic setup", the execution order of the given Assertions limits my use case. (JMeter adds the Assertions from "top to down".) By this I could concentrate on the Assertions, and avoid forget adding a second JSR223 Assertion.

For example an use case, that appears sometimes on my test cases: image

1 - Login is OK 1) Persist Error (as Assertion) is executed No error is persisted (as wanted)

2 - Login is NOK 1) Persist Error (as Assertion) is executed error is persisted (as wanted)

3 - Login is http 200 with wrong response body 1) Persist Error (as Assertion) is executed -> Nothing happens, since its http 200 1) Response Assertion is executed and fails No error is persisted (as needed)

4 - Login is http 200 with wrong response body 1) Persist Error (as Assertion) is executed -> Nothing happens, since its http 200 1) Response Assertion is executed and fails 1) Additonal Persist Error (as Assertion) is executed error is persisted (as wanted)

dgetzlaf commented 3 weeks ago

Hi @vlsi I think I found an obvious solution. Since I want to run groovy code after each Sampler (incl. after PostProcessor and Assertions), I can use a JSR223 Listener. (Script variables are also nearly the same)

What do you think about this?