eykrehbein / strest

⚡️ CI-ready tests for REST APIs configured in YAML
MIT License
1.74k stars 59 forks source link

Verify against returned arrays #12

Closed betsegaw closed 6 years ago

betsegaw commented 6 years ago

Say the returned response data has the following format,

[
  {
    "id": "5b9763b3ef7c4000018387cd",
    "from": "Bank",
    "to": "Bank",
    "amount": 0,
    "gameID": "SomePreDefiniedValue"
  }
]

How do I verify that "gameId" == "SomePreDefiniedValue"?

betsegaw commented 6 years ago

Apparently, this works.

validate:
      json:
        [0]:
          gameID: 'SomePreDefiniedValue'

Is this a hacky way of doing it or is there a proper way to do it?

eykrehbein commented 6 years ago

@betsegaw Unfortunately, there is no proper way to handle arrays, yet. Do you have any suggestions on how to improve this? 🤔

Paic commented 6 years ago

I think a good start for handling arrays would be to have the ability to define an item "template" like :

validate:
      json:
        [array]:
          name: Type(String)
          age: Type(Number)
          email: Type(String.Email)

Each item must match the template for the JSON to be validated.

I don't know if it's implementable but the Joi library has very advanced objects validation that would fit a lot of use case. 😄

Woops! Did not even see that Joi was already in the package.json and is already used in the lib 😁 I guess it's just a story of YAML <> Joi syntax

lchallis commented 6 years ago

Also interested in using this to validate arrays contain valid things, my response is like this :

{
    "d": [{
        "__type": "string",
        "boolValue": false,
        "descriptionString": "string",
        "name": "string",
        "Identifier": "string",
        "shortCode": "string",
        "shortName": "string",
        "statusId": "string"
    }]
}

So, the "d" object contains an array of objects of with the same properties. I'd like to validate that each of the objects in the array has the correct properties.

I'm able to validate that "d" is of type Array with this :

version: 1 
requests:
  getStatuses:
    url: http://domain.com/endpoint
    method: POST
    headers:
      "ApiKey" : "myApiKey"
    data:
      json: {"foo": "bar"}
    validate:
      json:
        d: Type(Array)
    log : true

But what I'd like to do is make sure each of the objects inside the "d" array is correct. Specifically I would like to avoid having to specify an array index identifier like [0] or whatever.

In the mean time, if anyone has any idea on the syntax I could use to check the first object in the "d" array I'm listening; would be good to have a further play.

betsegaw commented 6 years ago

@eykhagen I don't have too much of an opinion but here are the scenarios that are important from my point of view.

  1. Verify specific array element.
  2. Verify all array elements to make sure they match some value/condition.
    • This might be a bit more involved since in this case most likely you want to have a condition as opposed to a fixed value.
  3. Verify length of array.

I think this would cover a lot of scenarios where returned things are arrays.

betsegaw commented 6 years ago

P.S I really like the choice of YAML as the description of a test case. Yes, some test cases are very complex and require a lot of setup, but for the most part, 90% of basic verification tests are "check that f(x) = y" , so having a data only input like this helps simplify the whole thing. So thanks for putting this together!

jgroom33 commented 6 years ago

What if a joi validator is implemented:

version: 1 
requests:
  getStatuses:
    url: http://domain.com/endpoint
    method: POST
    data:
      json: {"foo": "bar"}
    validate:
      joi:
        ?????????

I don't know what the syntax beyond joi: would be, but this seems more explicit and potentially could handle more use cases.

jgroom33 commented 6 years ago

This should do what is noted in the original request

validate:
  jsonpath:
    $[?(@.amount==0)].amount: 0

It should be possible to use Value() in place of the 0's above.

GedersonChiquesi commented 6 years ago

My suggestion for this validation:

Validade:
   body.name: 'all names is the same'

and

validade:
  body.0.name: 'first name is the same'

For me, this way is so good for read and write.

jgroom33 commented 6 years ago

that is supported. Assuming .name is not an object. If .name is a string or number, then this functionality already exists: https://github.com/eykrehbein/strest/blob/master/tests/success/validate/jsonpath.strest.yml

If you are trying to validate an entire array or an object, then close this issue and open a new issue with that goal.

GedersonChiquesi commented 6 years ago

that is supported. Assuming .name is not an object. If .name is a string or number, then this functionality already exists: https://github.com/eykrehbein/strest/blob/master/tests/success/validate/jsonpath.strest.yml

If you are trying to validate an entire array or an object, then close this issue and open a new issue with that goal.

This is it. Tks