Modify CSLC validation script

[vbrancat]

The CSLC validation script performs a pixel-wise comparison of the difference between the real and imaginary parts of a reference and secondary products and fails when even one pixel is above a fixed threshold (1.0e-5).

This PR keeps the pixel-wise comparison but makes the validation script fail if the percentage of pixels for which the difference between the real and imaginary parts is above 0.1 %.

[vbrancat]

Thank you for your review @seongsujeong. Regarding of having the threshold exposed as an option, I am a bit conflicted about it. The idea behind the script is that we come up with something that makes the quality of the generated product to be acceptable. Having the threshold to be tunable would allow the users to make that choice (and eventually tune it to make the validation script pass). What would be the advantage of exposing it?

[seongsujeong]

Regarding of having the threshold exposed as an option, I am a bit conflicted about it. The idea behind the script is that we come up with something that makes the quality of the generated product to be acceptable. Having the threshold to be tunable would allow the users to make that choice (and eventually tune it to make the validation script pass). What would be the advantage of exposing it?

Sorry that my question was not so straightforward. I agree that the thresholds / criteria are fixed in the code. What I meant was that we declare some constants in the code (e.g. RELATIVE_TOL, MAX_RATIO_FAILED) and use them instead of the 1.0e-5 and 0.001 in the code.

[seongsujeong]

@vbrancat @hfattahi For more information about the difference, here I share the plot below. The plot shows the ratio of pixels (y axis) whose relative difference is larger than the value in the x axis. Left and right panel are for real and imaginary parts respectively.

The output from Docker on Aurora is the reference, and the one from Docker on NISAR server is the secondary.

The blue solid horizontal line is the relative tolerance, and the red solid vertical line is the threshold for the ratio of the pixels which did not pass the test.

Based on the plot, less than 0.0002% of the pixels have relative difference larger than 1.0e-5.

More information about the test data are as below: SAFE: S1A_IW_SLC__1SDV_20220501T015035_20220501T015102_043011_0522A4_42CC burst ID: t064_135524_iw2 Polarization: VV Orbit file : S1A_OPER_AUX_POEORB_OPOD_20220521T081912_V20220430T225942_20220502T005942.EOF

[vbrancat]

@LiangJYu do you have time to have a look at this script? @hfattahi if possible, can you make sure that the script does what we agreed for the validation of the CSLC-S1? Merci

[vbrancat]

@LiangJYu would it be fine for you to keep the current implementation? It seems to be clearer albeit more verbose

[LiangJYu]

@LiangJYu would it be fine for you to keep the current implementation? It seems to be clearer albeit more verbose

This PR fixes the bugs in my suggested code.

[vbrancat]

@seongsujeong and @LiangJYu this PR should be good to go. It would be great if you can have another look. Merci