Thomascountz / relay-service

API used by Relay text editor
0 stars 0 forks source link

Validate JSON schema #10

Closed Thomascountz closed 6 years ago

Thomascountz commented 6 years ago

When JSON is returned for IBM's Watson Tone Analyzer endpoint, validate the expected JSON schema and return meaningful errors.

This can be achieved using pattern matching

Thomascountz commented 6 years ago

Closes #8

Thomascountz commented 6 years ago

https://github.com/jonasschmidt/ex_json_schema

Thomascountz commented 6 years ago

I'm siding against validating the returned JSON from the Watson API for two reasons,

1) Their endpoints are versionized and their versioning pedagogy is well documented here: https://github.com/watson-developer-cloud/api-guidelines/#versioning. They consider an alteration to the JSON schema to be a breaking change, which would bump the version number.

2) The return structure can vary widely between one key and one value, to two top-level keys with several values, but never is the structure highly nested. At this point, the only validation that could be done is for the key document_tone and tone, all other keys and values returned are dependent on the results of the analysis.

As of version 2017-09-21, {"document_tone":{"tones":[]}} is the minimal return value from Watson for valid requests.

It would be my recommendation- if we believe this feature to provide necessary regression safety- to create a cron job that pings Watson's endpoint and validates the response against a known schema for that request. If this response ever failed validation, alerts could be generated for the development team.

Thomascountz commented 6 years ago

Look into automated contract testing

Thomascountz commented 6 years ago

A private postman monitor has been created to ping Watson's POST endpoint every day at noon. (This monitor is private, because team-level monitors aren't included on the free tier). This monitor runs three tests on two different requests.

The schema is validated with the tv4 library, which can be found here: https://github.com/geraintluff/tv4