#2159: adding Run Number into Standardization spark job

[lsulak]

Draft for now, I need to test a few things first & think about whether it should be part of Common Job Execution or only in the standardization.

Update: tests passed and it works as expected in the Menas UI & INFO file & Mongo DB.

--

I can see how the standardization and conformance are logically intertwined. The current implementation should generally work but there might be a case where Run Number is not present at all in the INFO file - if Conformation Job runs separately against dataset that has INFO file produced by Standardization Job that didn't produce the Run Number (because it is older than the one in this PR).

Q: Should the Run Number represent any standardization or conformance or combined job that ran against a dataset? If yes, then the current implementation is too naive and I'll add some check into the conformation job - something like: if the Run Number is not present in INFO file, add it.

[lsulak]

I also changed the implementation of the sorting of Additional Info data in Menas, it's now sorted by ascending order.

Before:

After:

[benedeki]

Q: Should the Run Number represent any standardization or conformance or combined job that ran against a dataset? If yes, then the current implementation is too naive and I'll add some check into the conformation job - something like: if the Run Number is not present in INFO file, add it.

I think it's good as it is now. No need to complicate it further.

[sonarcloud[bot]]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication