artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
8.01k stars 510 forks source link

Run `beforeScenario` only once before a scenario #842

Open rvarbanov opened 4 years ago

rvarbanov commented 4 years ago

I thought that I can use the beforeScenario to call my auth service to get an access token and use that for each call to the service I'm load testing. However, my auth function is called before each endpoint call. That is suboptimal because I don't want load test my auth service at the same time I load test all my other services.

Code example:

scenarios:
  - name: "Endpoint returns 200"
    beforeScenario: "auth"
    flow:
      - get:
          url: "/api/v1/endpoint"
          expect:
            - statusCode: 200
            - contentType: json
AnkitGuhe commented 3 years ago

@rvarbanov Wondering if you found the solution?

joberdick-rcd commented 3 years ago

Is there really not a way to do a one time call that excludes being ran by all virtual users?

hassy commented 3 years ago

beforeScenario is indeed meant to be run before each scenario execution (by each virtual user).

There is a way to run a call just once via a before section at the top level of the test definition. The contents of the section can be an arbitrary scenario, but any variables captured by that scenario will be available to all virtual users.

Quick example:

config:
  target: https://artillery.io
  phases:
    - duration: 10
      arrivalRate: 1

before:
  flow:
    - log: "running a request before virtual users run"
    # Grab a random URL from the homepage:
    - get:
        url: "/"
        capture:
          - selector: "a"
            attr: "href"
            index: "random"
            as: "url"

scenarios:
  # All virtual users will make requests to the same URL
  - flow:
      - log: "URL is {{ url }}"
      - get:
          url: "{{ url }}"
savitanagaraj commented 3 years ago

beforeScenario is indeed meant to be run before each scenario execution (by each virtual user).

There is a way to run a call just once via a before section at the top level of the test definition. The contents of the section can be an arbitrary scenario, but any variables captured by that scenario will be available to all virtual users.

Quick example:

config:
  target: https://artillery.io
  phases:
    - duration: 10
      arrivalRate: 1

before:
  flow:
    - log: "running a request before virtual users run"
    # Grab a random URL from the homepage:
    - get:
        url: "/"
        capture:
          - selector: "a"
            attr: "href"
            index: "random"
            as: "url"

scenarios:
  # All virtual users will make requests to the same URL
  - flow:
      - log: "URL is {{ url }}"
      - get:
          url: "{{ url }}"

@hassy I gave this a try. The exact same way. I had a post in the "before" section that could get me a bearer token, and I wanted to use that for the rest of my scenarios. I am getting an error. I have a bug logged for it #959 :

$ DEBUG=http:response artillery run Get_Token.yml
- TypeError: Cannot read property 'concat' of undefined
   at HttpEngine.createScenario (C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\core\lib\engine_http.js:91:23)
   at C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\core\lib\runner.js:511:37
   at new Promise (<anonymous>)
   at handleBeforeRequests (C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\core\lib\runner.js:501:10)
   at runner (C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\core\lib\runner.js:132:25)
   at Runner.run (C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\lib\runner-sp.js:60:3)
   at done (C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\lib\commands\run.js:191:14)
   at C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\node_modules\async\lib\async.js:52:16
   at C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\node_modules\async\lib\async.js:1209:30
   at done (C:\Users\snagaraj\AppData\Roaming\nvm\v12.20.1\node_modules\artillery\lib\commands\run.js:267:14)

My code is similar to this:


config:
  target: 'https://url1'
  phases:
   - name: "Default"
     duration: 1
     arrivalRate: 2
  payload:
    - path: "userlogin.csv"
      fields:
        - user1
        - user2
        - password1
        - password2
      order: "sequence"
before:
 - flow:
    #Get bearer token
   - log: "Getting token"
   - post:
      url: "https://url2/oauth2/v2.0/token"
      json:
       username: 'XXXX'
       password: 'XXXX'
       grant_type: 'password'
       client_id: '12345'
       client_secret: 'abcde'
       scope: 'openid
      capture:
       json: "$.id_token"
       as: "id_token"
      expect:
        - statusCode: 200
scenarios:
 - flow:
   - get:
       url: "url3"
       headers:
         authorization: 'Bearer {{id_token}}'
         user-login: "XXXX"
         user-role: "admin"
savitanagaraj commented 3 years ago

Looks like the before block works only with json files and not yaml.

AnkitGuhe commented 3 years ago

That's right before block only works with JSON file!

savitanagaraj commented 3 years ago

@AnkitGuhe it works with yaml too! @hassy responded to a bug I had logged. The contents of config.before should be an object rather than a list. That means, it should not be - flow, but flow. It works.

AnkitGuhe commented 3 years ago

I see, thanks for correcting @savitanagaraj, I had to rewrite all my tests in past and that was a pain. Appreciate it!

MarcoArredondo commented 3 years ago

Do you know if is there an "after" instruction too? I need to delete a parent element that I created in the "before" section. I looked through the documentation but I see nothing

I added this at the root level but seems to do nothing:

after:
  flow:
    - log: "Deleting project {{projectID}}"
    - delete: 
        url: /rest/v1/projects/{{projectID}}
Conrad6 commented 2 years ago

@savitanagaraj Can the beforeScenario be used to modify other properties like the headers, url, etc?

judaschwartz commented 2 years ago

I am trying to login Before running my flow (I do not want the login requests to be part of the scenario results I tried moving the login flow to the beforeScenario but that doesn't seem to run

config:
  target: '{{ $processEnvironment.URL }}'
scenarios:
  - name: 'Warm Up'
    beforeScenario:
      flow:
        - post:
            url: '/login'
            json:
              username: 'admin'
              password: 'password'
            capture:
              - json: '$.user._id'
                as: 'userId'
    flow:
      - post:
          url: '/api/graphql'
          json:
            - operationName: 'apiRequest'
              query: 'query aQuery($userId: ID!) { aQuery: testQuery(userId: $userId) { id name } }'
              variables:
                userId: '{{ userId }}'

Is there any way to achieve this?

MickL commented 2 years ago

@rvarbanov I think this issue can be closed as before and after are meant to be run only once per test as described in the documentation.

MickL commented 2 years ago

@judaschwartz may you explain the downvote? Isnt that exactly what was requested?

jackykwan-eventx commented 1 year ago

@judaschwartz may you explain the downvote? Isnt that exactly what was requested?

i think he need beforeScenario, not beforeTest beforeScenario is to create a global variable during the vuser session beforeTest is to create global variable during whole test Thats totally different

ashokyamsani commented 3 months ago

can we execute a function from processors to initialize page and browsers in before hook in config section