artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
7.85k stars 500 forks source link

Random data (from `fake-data`) doesn't work in `before` section #2756

Open RemiBardon opened 3 months ago

RemiBardon commented 3 months ago

Version info:

Artillery: 2.0.12
Node.js: v22.1.0
OS: darwin

Running this command:

DEBUG=http* artillery run init.yaml --solo

I expected to see this happen:

JSON body has "name" key.

Instead, this happened:

JSON body is empty (fake data not generated):

  http request: {
    "url": "http://127.0.0.1:8000/v1/workspace",
    "method": "PUT",
    "headers": {
      "user-agent": "Artillery (https://artillery.io)"
    },
-   "json": {
-     "name": "…"
-   }
+   "json": {}
  }

I suspect it's because

https://github.com/artilleryio/artillery/blob/88a6cf1cd94cd4e10e65836a1fe357ff842dbd7b/packages/artillery-plugin-fake-data/index.js#L41-L45

only maps scenarios in the scenarios section and not the ones in [before](https://www.artillery.io/docs/reference/test-script#before-and-after-sections) and [after](https://www.artillery.io/docs/reference/test-script#before-and-after-sections) sections.

Files being used:

config:
  target: http://127.0.0.1:8000
  plugins:
    fake-data: {}

before:
  flow:
    - put:
        url: "/v1/workspace"
        json:
          name: "{{ $randBrand() }}"

scenarios: []
hassy commented 3 months ago

thanks @RemiBardon! definitely looks like a bug /cc @bernardobridge

bernardobridge commented 3 months ago

👋 Thanks for the bug report @RemiBardon. I will take a look to see what can be done here

bernardobridge commented 2 months ago

Hey @RemiBardon 👋 ,

Apologies for the wait. This is a current limitation of before/after hooks, which we'll update the docs with. Unfortunately, it won't work with certain plugins that require adding functions to the scenario (e.g. fake-data or expect). Changing this behaviour is a little bit involved, as it requires refactoring how these hooks work, so we won't be able to tackle this just yet.

As a workaround, I would suggest using a javascript function from the before hook to accomplish the same. Something like:

const { randBrand } = require('@ngneat/falso');

function setRandomBrand(context, ee, next) {
    context.vars.randomBrand = randBrand();
    next();
};

module.exports = {
    setRandomBrand
}
config:
  target: http://127.0.0.1:8000
  processor: myProcessor.js

before:
  flow:
    - function: setRandomBrand
    - put:
        url: "/v1/workspace"
        json:
          name: "{{ randomBrand }}"

scenarios: []

Thanks again!