Open gshank opened 1 year ago
Specific case we're interested in:
constraints_enabled
(likely renaming this to contracted: true
)dbt_project.yml
, yaml file, within .sql
file)columns
are defined with name
+ data_type
. If any is missing (i.e. data_type
), raise a validation error during parsing.We can fall back to doing this not at parse time, since we'll catch it at runtime anyway. It's conceivable that we'd want this capability for other cases in the future, where we want to validate compatibility between configs/attributes that are being set in different places.
Currently we have an issue with snapshot config, where the validation requires that certain config attributes be set. Since all node configs are validated twice -- once at model parsing time and once at schema parsing time (if a schema config exists) -- if somebody attempts to set the snapshot config in a schema file it will fail on the model parsing step.
In a more recent initiative we have config that might be set in both/either model and schema file and are attempting to validate that all of the config is in place. If the feature is turned on in model config and the other config settings are in the schema file, the complete validation would fail on model parsing.
What we need to handle situations like this is to be able to flag certain validation situations to happen after all of the schema parsing has been done. For example if a snapshot node is constructed from a sql file but doesn't contain some of the required config, set a flag. If a schema file runs that validation and it succeeds, we can then turn the flag off. But if we reach the end of schema file parsing and the flag is still set, re-run validation and issue an error.