search

cloudera-labs / envelope

Build configuration-driven ETL pipelines on Apache Spark
Apache License 2.0
158 stars 89 forks source link

Spark Envelope job gets stuck in case an exception is thrown #29

Closed sgulati89 closed 6 years ago

sgulati89 commented 6 years ago

I was running Spark Envelope in 'Yarn Client mode' and I noticed that after config gets validated and job get submitted, if an exception is thrown from Spark Framework then Job gets stuck and does not exit.

I made a small change in 'com.cloudera.labs.envelope.run.Runner' class and it got fixed. I just added following piece of code in try and finally block:

try{
if (StepUtils.hasStreamingStep(steps)) {
      LOG.debug("Streaming step(s) identified");

      runStreaming(steps);
    }
    else {
      LOG.debug("No streaming steps identified");

      runBatch(steps);
    }
}
finally{    
    shutdownThreadPool();
}

Please check this issue.

jeremybeard commented 6 years ago

Hi @sgulati89, thanks for reporting that! We have also run into the same issue and we have almost exactly the same fix in place for the next release.

sgulati89 commented 6 years ago

Thanks @jeremybeard . If that is the case then you can close the issue. When is the next version release planned?

jeremybeard commented 6 years ago

At this stage we don't have a planned date or timeline, but we are actively working on it.