quantile-development / dagster-meltano

A Dagster plugin that allows you to run Meltano in Dagster
MIT License
41 stars 17 forks source link

cleaned up logging for errors #10

Closed jlloyd-widen closed 2 years ago

jlloyd-widen commented 2 years ago

@JulesHuisman I cleaned up the logging so that the exceptions would be controlled through Dagster instead of just the normal python channels. I also discovered a method for capturing Meltano tracebacks and passing them to the log that produces the step_failure. I'm not sure how robust it will be for every failure, but this is displaying how I'd like it to. Happy to hear feedback.