Closed DistributedDoge closed 8 months ago
Great idea! This project used to have dbt tests
at the begguining but was hard to integrate with Dagster.
I'm waiting for https://github.com/dagster-io/dagster/discussions/16527 to learn more about ways to integrate both projects together.
That said, a simple dbt test
won't hurt anyone!
Few config tweaks and interpreting dbt tests
as dagster asset checks
seems to be working as intended.
Can share recipie once I deal with #22 to place tests where they belong.
Nice! I was digging up into this and also realized that Dagster is integrating with dbt in a different way these days. Lets ship the tests with the current approach and then see if migrating is worth it.
dbt allows for
schema
testing, where we can declare what we want to see, and rundbt test
to see if reality conforms to expectations.I think it is worthwile to create those for some important models. Since we are already describing schema it would be nice to also:
addresses
so we know at glance if for exampleround_id
is actual address or just random string.is_lowercased
test for consistency.Ultimately, we could have a
dbt test
added as CI step to make sure that runs with malformed or missing assets do not proceed to IPFS upload stage.