snowplow / dataflow-runner

Run templatable playbooks of Hadoop/Spark/et al jobs on Amazon EMR
http://snowplowanalytics.com
19 stars 8 forks source link

Remove build dependency on Iglu Central schemas #79

Closed istreeter closed 2 years ago

istreeter commented 2 years ago

Until now, the build process has fetched cluster config and playbook config schemas from Iglu Central in order to auto-generate boiler plate code. But the schemas could just as easily live inside the repo. This will allow faster iteration of code changes, because it decouples Iglu Central releases and Dataflow Runner releases. We never validate the configs against the schema, so there is no real benefit of being strict about where the schema comes from.