artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
7.9k stars 504 forks source link

Artillery stops execution if any json response property value is an empty string #822

Open himtar12 opened 4 years ago

himtar12 commented 4 years ago

Consider below case:

config:
target: "https://my.app.local"
phases:
- duration: 600
arrivalRate: 10
variables:
productIds:
- ["id1", "id2", "id3"]
scenarios:

flow:
loop:
get:
url: "/products/{{ $loopElement }}"
capture:

json: $[0].id
as: prod1
json: $[0].link.id
as: prod2
get:
url: "/element/{{ prod2}}"
over: productIds

If "$[0].link.id" property have an empty string which should set variable "prod2" as an empty string then artillery will never execute the second GET request since the execution will get stuck on the first GET request because it found an empty string for "$[0].link.id".

himtar12 commented 3 years ago

@hassy Any comments?

rmjwilbur commented 1 year ago

@himtar12 and @hassy - I get the same behaviour in 2.0.0-32 when a property doesn't exist. For example:

          capture:
            - json: '$.oops'
              as: oops
          expect:
            - statusCode: 200
      - log: 'I should see this'

When using debug, it appear as:

  http   oops: { value: undefined, strict: undefined, failed: true }

and

      ok statusCode 200 

I don't see any error, but execution seems to stop there. I don't even see the log line after it.

Out of curiosity, I tried this with properties having values of 0, null, and an empty string. They all worked fine.

I test the output for an expected final log line, so I know my test failed somehow, but this can be a bit challenging to figure out.