artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
7.88k stars 501 forks source link

'capture' keyword doesn't seem working with 'before' keyword #3283

Closed bear-su closed 1 month ago

bear-su commented 1 month ago

Version info: 2.0.18 Node.js: 22.2.0

Running this command:

artillery run test.yml

I expected to see this happen:

I expected to get token from the request to /auth.

Instead, this happened:

/Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:673
                    return done(new Error('Failed capture or match'), context);
                                ^

Error: Failed capture or match
    at /Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:673:33
    at /Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:473:16
    at replenish (/Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:1009:25)
    at /Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:1019:9
    at eachLimit$1 (/Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:3199:24)
    at Object.<anonymous> (/Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:1049:16)
    at captured (/Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:628:21)
    at /Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/@artilleryio/int-commons/engine_util.js:499:16
    at /Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:473:16
    at replenish (/Users/user/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:1009:25)

Files being used:

config:
  target: 'http://localhost:9099' # 테스트 진행할 URL
  phases:
    - duration: 50
      arrivalRate: 3
  payload:
    - path: './data/article.csv'
      fields:
        - 'title'
        - 'content'
        - 'author'
        - 'domesticStockArticleType'

before:
  flow:
    - log: 'Get auth token'
    - post:
        url: '/auth'
        json:
          username: 'arto'
          password: 'localpassword'
        capture:
          - json: $.data.token
            as: token

scenarios:
  - flow:
      - post:
          url: '/admin/articles/recommendations'
          headers:
            authorization: 'Bearer {{ token }}'
          json:
            article:
              - {{ title }}
              - {{ content }}
              - {{ author }}
              - {{ domesticStockArticleType }}

[ For more information ]

bernardobridge commented 1 month ago

Hi @bear-su ,

I just created a server that modelled that response, and I was able to get the capture working in the before section just fine. Are you sure your response is being returned correctly from the API? It might be worth trying to log the response from the API using an afterResponse hook

This is what I used:

const express = require('express');
const bodyParser = require('body-parser');
const jwt = require('jsonwebtoken');

const app = express();
const port = 9099;

app.use(bodyParser.json());

app.post('/auth', (req, res) => {
  const adminData = {
    id: 7,
    adminId: "bears",
    name: "",
    introduction: "",
    profileImageUrl: "",
    adminRole: "",
    createdBy: 1,
    updatedBy: 1,
    createdAt: new Date().toISOString(),
    updatedAt: new Date().toISOString()
  };

  const token = jwt.sign(
    {
      admin_role: "abc",
      admin_id: "asdfasdf",
      name: "asfas",
      id: "7",
      exp: Math.floor(Date.now() / 1000) + (60 * 60) // 1 hour expiration
    },
    'your-secret-key',
    { algorithm: 'HS512' }
  );

  res.json({
    status: "OK",
    data: {
      admin: adminData,
      token: token
    },
    message: null
  });
});

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});
config:
  target: 'http://localhost:9099'
  phases:
    - duration: 5
      arrivalRate: 5

before:
  flow:
    - log: 'Get auth token'
    - post:
        url: '/auth'
        json:
          username: 'arto'
          password: 'localpassword'
        capture:
          - json: $.data.token
            as: token

scenarios:
  - flow:
      - log: "Token is {{ token }}"
bear-su commented 1 month ago

@bernardobridge Hi ! I closely checked it again. It still shows the same error.

I remove capture keyword and tried running. It runs well and shows "Token is undefined. "

config:
  target: 'http://localhost:9099'
  phases:
    - duration: 5
      arrivalRate: 5

before:
  flow:
    - log: 'Get auth token'
    - post:
        url: '/auth'
        json:
          username: 'arto'
          password: 'localpassword'

scenarios:
  - flow:
      - log: "Token is {{ token }}"
bernardobridge commented 1 month ago

Hello @bear-su ,

I still can't reproduce the error with the test server example I put above... It works fine in a before hook.

Perhaps you can inspect the response that's coming from the post and see if anything stands out as wrong. You can attach an afterResponse hook to check that:

config:
  target: 'http://localhost:9099'
  phases:
    - duration: 5
      arrivalRate: 5
  processor: ./helpers.js

before:
  flow:
    - log: 'Get auth token'
    - post:
        url: '/auth'
        json:
          username: 'arto'
          password: 'localpassword'
        afterResponse: logResponse

scenarios:
  - flow:
      - log: "Token is {{ token }}"

And create a ./helpers.js file:

function logResponse(req, res, context, ee, next) {
    console.log(`Response body is:`)
    console.log(res.body);
    console.log(`Status Code is: ${res.statusCode}`);

    next();
};

module.exports = {
    logResponse
}
bear-su commented 1 month ago

@bernardobridge I tried it and share the result.

[.yml]

config:  
  target: 'http://localhost:9099' # 테스트 진행할 URL  phases:  
    - duration: 1  
      arrivalRate: 1  
  processor: helper.js  

before:  
  flow:  
    - post:  
        url: '/admins/login'  
        json:  
          username: 'arto'  
          password: 'localpassword'  
        capture:  
          - json: $.data.token  
            as: token  

scenarios:  
  - flow:  
      - post:  
          url: '/admin/articles/recommendations'  
          headers:  
            authorization: 'Bearer {{ token }}'  
                - post:
          url: '/admin/articles/recommendations'
          headers:
            authorization: 'Bearer {{ token }}'
          json:
            article:
              - {{ title }}
              - {{ content }}
              - {{ author }}
              - {{ domesticStockArticleType }}
          afterResponse: 'logResponse'

[helper.js]

function logResponse(req, res, context, ee, next) {  
    console.log("This is test")  
    console.log(`Response body is:`)  
    console.log(res.body);  
    console.log(`Status Code is: ${res.statusCode}`);  

    next();  
};  

module.exports = {  
    logResponse  
}

[console.log] DEBUG=http artillery run test/domestic-stock-register-test.yml

Test run id: t7apy_z5bqjd9dpre6m4y77xjwkcxf8mkf8_64mj
⠋   http request: {
  "url": "http://localhost:9099/admins/login",
  "method": "POST",
  "headers": {
    "user-agent": "Artillery (https://artillery.io)"
  },
  "json": {
    "username": "arto",
    "password": "localpassword"
  }
} +0ms
  http captures and matches: +1ms
  http {} +0ms
  http { token: { value: undefined, strict: undefined, failed: true } } +1ms
/Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:677
                    return done(new Error('Failed capture or match'), context);
                                ^

Error: Failed capture or match
    at /Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:677:33
    at /Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:473:16
    at replenish (/Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:1009:25)
    at /Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:1019:9
    at eachLimit$1 (/Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:3199:24)
    at Object.<anonymous> (/Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:1049:16)
    at captured (/Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/engine_http.js:632:21)
    at /Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/@artilleryio/int-commons/engine_util.js:499:16
    at /Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:473:16
    at replenish (/Users/arto/.nvm/versions/node/v22.2.0/lib/node_modules/artillery/node_modules/async/dist/async.js:1009:25)
bear-su commented 1 month ago

I dig into the problem and I found the problem. It doesn't catch it as an error since I use custom API Response which is this.

@Getter
@Builder
public final class APIResponse<T> {
    private final HttpStatus status;
    private final T data;
    private final String message;
}

So i added the debug log in engine_http.js

if (process.env.DEBUG) {
  debug("Captured: {}", resForCapture);
}