eykrehbein / strest

⚡️ CI-ready tests for REST APIs configured in YAML
MIT License
1.74k stars 59 forks source link

Allow for validating undefined / non-existent properties in responses #138

Open larrybotha opened 5 years ago

larrybotha commented 5 years ago

For some requests it's useful to validate that a value was not returned, e.g. when creating a user and ensuring the response doesn't have the password.

At the moment https://github.com/eykrehbein/strest/blob/master/src/test.ts#L426 prevents one from testing non-existence of properties.

version: 2
requests:
  create_user:
    request:
      url: example.com/user
      method: POST
        mimeType: 'application/json'
        text:
           email: 'test@example.com'
           password: 'password'
      validate:
        - jsonpath: content.email
          expect: 'test@example.com'
        - jsonpath: content.password
          expect: undefined

Describe the solution you'd like Adding undefined as a 'type' or value to validate against.

Describe alternatives you've considered Adding null to responses is a work-around, but isn't ideal.

jgroom33 commented 5 years ago

Another possible workaround (and possibly a better approach for some circumstances) is to use jsonschema validation. I haven't tried it, but a quick search makes it looks like it's possible to validate jsonschema using null and undefined. It would be necessary to pass content instead of content.password to the jsonschema validation

larrybotha commented 5 years ago

Managed to use additionalProperties as a workaround for now:

version: 2

variables:
  endpoint: users
  password: 'password'
  schemaValidate:
      properties:
        email:
          type: [string]
      required:
        - email
      additionalProperties: false

requests:
  create_user:
    request:
      url: example.com/user
      method: POST
        mimeType: 'application/json'
        text:
           email: 'test@example.com'
           password: 'password'
      validate:
        - jsonpath: content.email
          expect: 'test@example.com'

additionalProperties: false errors if there are any additional properties, and doesn't specify which specifically, so there's some cognitive overhead in trying to determine which property in the response is responsible for the error.

I also attempted using:

variables:
  endpoint: users
  password: 'password'
  schemaValidate:
      properties:
        email:
          type: [string]
      required:
        - email
      additionalProperties: 
        password:
            type: [undefined]

but without any luck. It seems as through additionalProperties expects an array of types, anyhow.


With a quick evaluation of using [undefined] for the password property's type in schemaValidate it seems as though any values in the response are ignored, whether undefined or defined.


EDIT:

It would be necessary to pass content instead of content.password to the jsonschema validation

Still need to give this a bash

EDIT #2: Already done this ^, as per my schemaValidate example :P