SendGrid webhook: make schema configurable

[benjben]

Context

network/json-schema-validator library uses a manually-maintained list of TLDs to validate the emails, which can cause the emails sent via SendGrid webhook to fail validation because their domain is not known.

A possible solution is to make the schema hard coded in Sendgrid adapter configurable.

Guide

Enrich has kind of 3 steps:

• loaders: deserialize the Thrift payload
• adapters: take the info from the HTTP request (~ collector payload) to create a unified CollectorPayload, that has the same format wherever the source (webhook, Snowplow event..)
• enrichments: this is where the validation of the input event happens, as well as the actual enrichments

The configuration format (for example for Kinesis) is defined in here. We probably want to add a new section that could look like this :

"adapters": {
  "sendgrid": {
    "schema": "iglu://..."
  }
}

There are several sbt modules:

• common contains the adapters and enrichments logic (we'll need to update it so that the adapters can become configurable)
• common-fs2 contains the logic related to configuring the app, reading events, writing them (we'll need to update it so that the config is propagated to the adapters)
• kinesis: logic related to reading from/writing to Kinesis (we won't need to touch it)
• ...

When running enrich-kinesis for instance, the starting point is here. It then goes here and then goes to common-fs2's Run here. It creates an Environment here. In our case we'll need to take the schema defined in the config and pass it to the AdapterRegistry here when creating the Environment