Add documentation for integration testing and OutputCheckers interface

[mindyls]

Requested Update

Description

• Integration testing in GeoIPS involves storing the "correct" output in test comparison directories, and comparing the current output against those known outputs during processing. GeoIPS run_procflow knows to implement the output checkers comparison tests if the "--compare_path" argument is passed with a path to a directory of all expected outputs for the current run.
• When the output checkers flag an output as either bad (different between test comparison directory and current output path, but same file name), missing comparison (current output product that was NOT found in test comparison directory), or missing product (file in test comparison directory that was NOT found in current output), then if those failed comparisons were expected / intentional, we must update the test comaprison outputs appropriately so that the next run will pass.

• At runtime when --compare_path is passed, GeoIPS produces text files with the rm and cp commands required to appropriately update the test comparison files. This entire process needs to be fully documented.

  • source rm_MISSINGPRODUCTS.txt (Removes all files from the test comparison directory that were NOT found in the current run but WERE found in the test comparison directory)
  • source cp_MISSINGCOMPARE.txt (Copies all files into the test comparison directory from the current run that were NOT found in the test comparison directory but WERE found in the product output of the current run.
  • source cp_BADCOMPARE.txt (Copies all files from the current run to the test comparison directory that did not match exactly between current output product and test comparison file)

Background and Motivation

• This has never been documented. Ensure the integration test process is well documented.

Alternative Solutions

• [ OPTIONAL: If appropriate, list alternative fixes that could be considered]

Environment

• geoips >= 1.12.0 (do not update until the 1.12.0 release)

Code to demonstrate issue

• [ IF APPROPRIATE: Can include steps here to identify/demonstrate bug fix or desired feature ]

Checklist for Completion

• [ ] <!--- Include a list of steps that must be complete in order to close this Issue --->

[evrose54]

This is partially completed via #491. Covers output_checkers section, but not integration testing.

[biosafetylvl5]

Parent: #531