search

artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
8.06k stars 511 forks source link

Provide an alias for multiple expect.equals to distinguish them #3408

Open ash211 opened 1 week ago

ash211 commented 1 week ago

Version info: 

Artillery: 2.0.21
Node.js:   v22.9.0
OS:        darwin

Running this command: 

artillery run artillery-load-test.yml

with this config in artillery-load-test.yml:

<snip>
scenarios:
  - flow:
      - function: 'loadPayload'
      - post:
          url: '/my-url'
          json: '{{ payload }}'
          capture:
            - json: "$.executionResult.type"
              as: resultType
            - json: "$.executionResult.success.returnValue.integer"
              as: returnValueInt
          expect:
            - statusCode: 200
            - contentType: json
            - equals:
              - '{{ resultType }}'
              - 'success'
            - equals:
              - '{{ returnValueInt }}'
              - '42'

I'm looking at this response:

plugins.expect.failed: ......................................................... 167216
plugins.expect.failed.contentType: ............................................. 41591
plugins.expect.failed.equals: .................................................. 84034 ***
plugins.expect.failed.statusCode: .............................................. 41591
plugins.expect.ok: ............................................................. 3392
plugins.expect.ok.contentType: ................................................. 1061
plugins.expect.ok.equals: ...................................................... 1270 ***
plugins.expect.ok.statusCode: .................................................. 1061

I have two equals expressions, but they are both counting against the numbers at plugins.expect.failed.equals and plugins.expect.ok.equals. There's no way to tell if these two equals expressions are failing at the same frequency, or different. It would be helpful if there was a way to distinguish between these.

Desired output would be a way to get an output like this:

plugins.expect.failed: ......................................................... 167216
plugins.expect.failed.contentType: ............................................. 41591
plugins.expect.failed.equals.resultType: ....................................... 42017 ***
plugins.expect.failed.equals.returnValueInt: ................................... 42017 ***
plugins.expect.failed.statusCode: .............................................. 41591
plugins.expect.ok: ............................................................. 3392
plugins.expect.ok.contentType: ................................................. 1061
plugins.expect.ok.equals.resultType: ........................................... 635 ***
plugins.expect.ok.equals.returnValueInt: ....................................... 635 ***
plugins.expect.ok.statusCode: .................................................. 1061

where the equals.resultType and equals.returnValueInt are counted independently.