datasets-br / sql-unifier

Try all datasets in a single SQL table, plug-and-play!
3 stars 0 forks source link

Assert, as validation and constraint strategies #12

Open ppKrauss opened 6 years ago

ppKrauss commented 6 years ago

As in the Schematron-validation philosophy... To validate any kind of constraint, but, mainlly, to validate frictionlessdata.io/table-schema/constraints


3-Layer architecture:

  1. Git with datapackage, CSVs, etc. The main reference and data origin... The single source of truth in the community context.
  2. The datasets on dataset.big, as single source of truth in the database context.
  3. Views, materialized views and tables for performance (fast queries) or for constraint-control in maintenance operations (validating new datasets or consolidating the ecosystem).
ppKrauss commented 6 years ago

For complex asserts, suggestion is TAP standard, "Test Anything Protocol". See http://pgtap.org/

For simple asserts, see pg9.5+ https://postgres.cz/wiki/PL/pgSQL_(en)#ASSERT_statement