search

AbsaOSS / enceladus

Dynamic Conformance Engine
Apache License 2.0
29 stars 14 forks source link

#2165 Sparkjobs runnable multiple times #2166

Closed dk1844 closed 1 year ago

dk1844 commented 1 year ago

This PR directly reacts to problems outlined in #2165, i.e. inability to run multiple spark jobs within a single application ("as a library").

Specifically: Atum's spark.disableControlMeasuresTracking() is now called at the end of each Std/Conf/Std+Conf spark job (instead of relying on control measure tracking being closed by closing related spark session)

Organization wise - prepareStandardization() and finishJob() are now paired in this sense

A related unit test added.

Closes #2165

sonarcloud[bot] commented 1 year ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication