GoogleCloudPlatform / cloud-ops-sandbox

Cloud Operations Sandbox is an open source collection of tools that helps practitioners to learn O11y and R9y practices from Google and apply them using Cloud Operations suite of tools.
Apache License 2.0
406 stars 148 forks source link

tests: Add integration tests for validating SRE Recipe config schema #836

Closed Gan-Tu closed 3 years ago

Gan-Tu commented 3 years ago

The validation mainly relies on the newest JSON Schema (https://json-schema.org) definition, and it works by first converting the YAML config to JSON, and then validate directly on the converted JSON. Added relevant README changes as well.

In order to make decoupling JSON schema easier, I decided to write a custom schema bundling logic for schema "references", since the Python version of the JSON schema validator doesn't support referencing local files well.

github-actions[bot] commented 3 years ago

Open in Cloud Shell You can also use the Stage Website Action if there were updates to the website.

Note: Open in Cloud Shell may not work properly if this PR contains changes to the custom Cloud Shell image

daniel-sanche commented 3 years ago

So to make sure I'm reading this correctly: the purpose of this PR is to add a test to ensure recipes are valid by defining the expected JSON schema, and adding a test to check that all recipe YAML conforms to it?

And going forward, if we want to build the recipe codebase (eg, adding actions), we have to adjust the scemas, but when adding future recipes we don't have to touch anything?

Let me know if I have any misconceptions. Thanks!

Gan-Tu commented 3 years ago

So to make sure I'm reading this correctly: the purpose of this PR is to add a test to ensure recipes are valid by defining the expected JSON schema, and adding a test to check that all recipe YAML conforms to it?

And going forward, if we want to build the recipe codebase (eg, adding actions), we have to adjust the scemas, but when adding future recipes we don't have to touch anything?

Let me know if I have any misconceptions. Thanks!

Your understanding is correct. It ensures the YAML schema is followed, but no adjustment needed for new recipes, unless new actions are defined.

Gan-Tu commented 3 years ago

Moved test to CI