#2163 Loading of RecordIdGeneration settings fixed for Std/Std&Conf -…

AbsaOSS / enceladus

Dynamic Conformance Engine

Apache License 2.0

30 stars 14 forks source link

#2163 Loading of RecordIdGeneration settings fixed for Std/Std&Conf -… #2164

Closed dk1844 closed 1 year ago

dk1844 commented 1 year ago

This PR attempts to fix #2163

Loading of RecordIdGeneration settings fixed for Std/Std&Conf (extracted to a single method in StandardizationExecution)
RecordIdGeneration from spark-data-standardization library reused, local copy removed.

@miroslavpojer please test

Closes #2163

sonarcloud[bot] commented 1 year ago

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

miroslavpojer commented 1 year ago

[x] code reviewed
[x] pulled
[x] built
[x] run

Problem tested on Enceladus's Hermes test. (Where problem was observed.) No Unit or Integration tests exist for this topic. Periodical run of Hermes tests is required.

miroslavpojer commented 1 year ago

[x] code reviewed

[ ] pulled

[ ] built

[ ] run

Not just seems to fix the issue, but improves the code too.

Would only check if the id is not generated twice if Standardization and Conformance are run separately. 🤔

I did test were Standardization and Conformance are done as separated spark-submit calls.