search

NASA-IMPACT / veda-pforge-job-runner

Apache Beam + EMR Serverless Job Runner for Pangeo Forge Recipes
2 stars 2 forks source link

Flink Swallows Pipeline Errors #27

Closed ranchodeluxe closed 6 months ago

ranchodeluxe commented 7 months ago

Problem

LocalDirectBakery experience will show where/when things go wrong in pipelines. For recent recipe runs it show that we generate a ton of botocore.exceptions.ConnectionError subclasses during runs. Flink does not show us any of this information for some reason 😞

Possible Solution

No idea, but we need to figure this out or we won't have any good information about why pipelines fail

ranchodeluxe commented 7 months ago

Tim Hodson at USGS is seeing the same thing for a data point

ranchodeluxe commented 6 months ago

closing this and moving here: https://github.com/pangeo-forge/pangeo-forge-cloud-federation/issues/20