Update to new config system

dagster-io / hooli-data-eng-pipelines

Example Dagster Cloud code for the Hooli Data Engineering organization.

75 stars 15 forks source link

Update to new config system #20

Closed slopp closed 1 year ago

slopp commented 1 year ago

Uses the new resource system in both custom resources and integrations. Also refactors to take advantage of resources being supplied directly to sensors.

For examples of custom resources look at the EmailAlert and FileSystem classes.

For integration examples, look at the main definitions file where all resources are defined:

[x] DuckDB
[x] Snowflake
[ ] Databricks
[x] dbt cli resource
[x] AWS - pickle IO manager / s3 resource

github-actions[bot] commented 1 year ago

Your pull request is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
`data-eng-pipeline`		View in Cloud	May 08, 2023 at 02:21 PM (UTC)

github-actions[bot] commented 1 year ago

Your pull request is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
`basics`		View in Cloud	May 08, 2023 at 02:15 PM (UTC)