search

artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
8.02k stars 510 forks source link

Execution Time #916

Open n1architect opened 3 years ago

n1architect commented 3 years ago

Hi,

When I set this config:

      duration: 100
      arrivalRate: 150
      maxVusers: 20000

I expect this it will run for 100 seconds, but it keeps running for 20 minutes and hasn't yet finished.

Is it the issue or I'm missing on something?

hassy commented 3 years ago

An arrival phase is not the same as the runtime of a test.

https://artillery.io/docs/guides/getting-started/core-concepts.html#Example-How-long-will-a-test-run-for

To use a made-up real world example. You're load testing a supermarket by sending in shoppers. 10 new customers enter the shop every second, and each one of them has a list of things to buy. You're going to be sending in 10 new shoppers every second for 60 seconds. These are your load phases and arrival rates.

The amount of time each customer will spend in the shop will depend on the length of their shopping list (i.e. the scenario), and the duration of their visit will depend on many other factors, such as congestion in aisles, in front of certain shelves, or at checkouts. These are like bottlenecks in the service you're testing which become visible as load increases.

At second 60 the last 10 customers enter the shop. Does that mean the load test is over? No. The last 10 customers will need time to complete their shopping, and who knows how many earlier customers are still in the shop. The test finishes when the last customer exits, with or without all of their shopping (e.g. a connection was reset because of high load).

Hope that makes sense.

dirtyren commented 10 months ago

Hey @hassy , I read your explanation for the duration and all but when using the HLS plugin, the test will run for the time specify on the duration, bandwidth will go to zero so will the vusers, but artillery will keep running on forever. I think there is something odd about the way duration is working today, because even with simples http tests using 1 vUser the test will run many times longer the duration. Tks.