search

dagster-io / hooli-data-eng-pipelines

Example Dagster Cloud code for the Hooli Data Engineering organization.
72 stars 15 forks source link

DRAFT: testing freshness check from dbt metadata #96

Closed slopp closed 2 months ago

slopp commented 2 months ago

Needs to land after https://github.com/dagster-io/dagster/pull/22290

~Main question, @dpeng817 , do I also need to create a sensor for these checks?~

github-actions[bot] commented 2 months ago

Your pull request at commit 637ce5d2262137186e73ae4a2565e84fd6578829 is automatically being deployed to Dagster Cloud.

Location Status Link Updated
demo_assets Deploy failed Jun 10, 2024 at 05:48 PM (UTC)
basics Deploy failed Jun 10, 2024 at 05:48 PM (UTC)
batch_enrichment Deploy failed Jun 10, 2024 at 05:48 PM (UTC)
data-eng-pipeline Deploy failed Jun 10, 2024 at 05:48 PM (UTC)
snowflake_insights Deploy failed Jun 10, 2024 at 05:48 PM (UTC)
dpeng817 commented 2 months ago

@slopp yea, you need to create a sensor.

dpeng817 commented 2 months ago

You need to define the actual freshness SLA here as well: https://github.com/dagster-io/hooli-data-eng-pipelines/blob/master/dbt_project/models/schema.yml

slopp commented 2 months ago

Its alive

@cnolanminich I'd appreciate your thoughts on where I put the meta section given our dbt setup

Screen Shot 2024-06-10 at 1 30 18 PM
github-actions[bot] commented 2 months ago

Your pull request at commit e74ee15eaeb545fbc90b244107414435be99289c is automatically being deployed to Dagster Cloud.

Location Status Link Updated
demo_assets Deploy failed Jun 10, 2024 at 07:50 PM (UTC)
basics Deploy failed Jun 10, 2024 at 07:50 PM (UTC)
batch_enrichment Deploy failed Jun 10, 2024 at 07:50 PM (UTC)
data-eng-pipeline Deploy failed Jun 10, 2024 at 07:50 PM (UTC)
snowflake_insights Deploy failed Jun 10, 2024 at 07:50 PM (UTC)