Working on CIP Checks Framework design

[yodahekinsew]

Design goals

• Fix current error logging system by making it more human-readable
• Implement general interface for adding future promoter checks
• Add promoter check to reject Pull Requests that attempt to remove images from promoter manifests

[yodahekinsew]

Implementation Plan

1. First create the general check function interface

   • Add test for the general check function. The general check function will take in an array of PullChecks and iterate through them, and should return an error if any of the PullChecks return an error. A simple unit test can be used for this check function by creating two anonymous PullChecks (one that always returns nil, and one that always returns an error), and then passing these functions into the general function to see if the expected output occurs
   • After the test is done, we can implement the interface and the general function

2. Second fix the current error logging system

   • Update the current logging to listen to the verbosity tag
   • Add a test for the JSON summary function.
   • After the test is added, create the JSON summary function and a JSON summary boolean tag. If the JSON summary boolean tag is enabled, then don't log any errors, warnings, or info like normal; instead just collect them and log them as a json whenever the job exits in the json summary function

3. Once the above are done, we can implement the image removal check

   • Add test for the image removal check. The basic unit tests just need to consist of two sets of promotion edges where an edge is missing from the "pull request" set. The image removal PullCheck function should be able to detect that an image was deleted between those two sets. Should also add unit tests for edge cases like both sets have the same destination images but some of the source images have been changed.
   • After the unit test is added, then implement the PullCheck function. The pull request manifest is parsed and read at the beginning of the CIP binary, so in the PullCheck we can use go-git to checkout the master branch and generate the master branch's promotion edges. It might be better to clone instead of checking out just in case someone wants to use the pull request directory for something in another PullCheck. The PullCheck then spans the two promotion edge sets
   • Implement an end to end test