dagster-io / hooli-data-eng-pipelines

Example Dagster Cloud code for the Hooli Data Engineering organization.
72 stars 15 forks source link

add local dev and update docs for Sling #75

Closed cnolanminich closed 4 months ago

cnolanminich commented 5 months ago

the Sling loader was failing in prod, and I wanted to debug it. There must have been an update to Sling between version upgrades because we needed to change the AWS_BUCKET from s3://{bucket_name}/{folder_path} (old) to just {bucket_name}. That was an env var change which is now live, but for me to figure that out I had to get the local version working. This PR adds that local support:

github-actions[bot] commented 5 months ago

Your pull request is automatically being deployed to Dagster Cloud.

Location Status Link Updated
batch_enrichment View in Cloud Apr 09, 2024 at 08:23 PM (UTC)
data-eng-pipeline View in Cloud Apr 09, 2024 at 08:23 PM (UTC)
snowflake_insights View in Cloud Apr 09, 2024 at 08:23 PM (UTC)
basics View in Cloud Apr 09, 2024 at 08:23 PM (UTC)
demo_assets View in Cloud Apr 09, 2024 at 08:23 PM (UTC)