search

sodadata / soda-sql

Soda SQL and Soda Spark have been deprecated and replaced by Soda Core. docs.soda.io/soda-core/overview.html
https://docs.soda.io/
Apache License 2.0
59 stars 16 forks source link

Maybe revise the hard key access for `source` nodes in dbt integration #189

Closed bastienboutonnet closed 2 years ago

bastienboutonnet commented 2 years ago

Describe the bug If a manifest.json does not have nodes of type sources the parsing can throw a key error:

11:40:11.599 INFO  SodaSqlCon | STDERR:   File "/usr/local/lib/python3.8/site-packages/sodasql/dbt.py", line 83, in parse_manifest
11:40:11.600 INFO  SodaSqlCon | STDERR:     for source_name, source in manifest["sources"].items()
11:40:11.600 INFO  SodaSqlCon | STDERR: KeyError: 'sources'

We might want to do a "soft" access via a .get() if possible.

I didn't do further research into how feasible this is but if no one picks this up I'll have a closer look as soon as I have a minute

The issue was highlighted by @Antoninj who was writing some integration tests and had an outdated test file which did not contain any source nodes.