search

snowplow / schema-ddl

ASTs and generators for producing various DDL and Schema formats
Apache License 2.0
5 stars 4 forks source link

Release/0.22.3 #204

Open pondzix opened 8 months ago

coveralls commented 8 months ago

Pull Request Test Coverage Report for Build 8435310588

Details

Totals Coverage Status
Change from base Build 8434104603: 0.1%
Covered Lines: 1250
Relevant Lines: 1549
💛 - Coveralls
istreeter commented 8 months ago

By the way, you might be wondering... why would a loader receive invalid data anyway, if it has already been validated by Enrich.

There are two scenarios that I know of:

  1. The user has patched a schema in Iglu. So Enrich validated it against the old schema, and the loader tries to load it according to the new schema. I have no sympathy for this scenario! Anything we do in this scenario is ok, as long as the loader does not break
  2. The data falls into one of the edge-case categories described over here

If you understand the second scenario, it might help guide you on what is the correct end behaviour.