Run this after a change to the codebase to ensure debug output hasn't changed.
Intended usage usage:
Run python runme.py <reference_output> on the version of the code before the changes
Run python runme.py <target_output>on the version of the code after the changes
From the test/ directory, run python outputvalidation.py <opts> ../reference_output.csv ../target_output.csv
Prints YARP if the files match as expected, otherwise prints NARP
outputvalidation.py will ignore new columns in the target_output (that do not appear in reference_output) CSV automatically. If you expect any columns to be different across the two output files, specify them after the --ignore flag.
Currently outputvalidation.py does not support intentionally removing variables from the report output, but this could be added easily. That is to say if a variable appears in reference_output.csv but not target_output.csv, this will yield NARP
Testing Note: if you intend to test the functionality of this code by modifying output CSVs by hand in
Excel, make sure you open and save both files from Excel. Simply opening one of them and pressing save will change the way the data is formatted, causing outputvalidation.py to return NARP
A quick-and-dirty output validator.
Run this after a change to the codebase to ensure debug output hasn't changed. Intended usage usage:
python runme.py <reference_output>on the version of the code before the changespython runme.py <target_output>on the version of the code after the changestest/directory, runpython outputvalidation.py <opts> ../reference_output.csv ../target_output.csvYARPif the files match as expected, otherwise printsNARPoutputvalidation.pywill ignore new columns in thetarget_output(that do not appear inreference_output) CSV automatically. If you expect any columns to be different across the two output files, specify them after the--ignoreflag.Currently
outputvalidation.pydoes not support intentionally removing variables from the report output, but this could be added easily. That is to say if a variable appears inreference_output.csvbut nottarget_output.csv, this will yieldNARPTesting Note: if you intend to test the functionality of this code by modifying output CSVs by hand in Excel, make sure you open and save both files from Excel. Simply opening one of them and pressing save will change the way the data is formatted, causing
outputvalidation.pyto returnNARP