A tool for estimating the future energy use, carbon emissions, and capital and operating cost impacts of energy efficiency and demand flexibility technologies in the U.S. residential and commercial building sectors.
Introduces a class to compare integration test results on a branch with the results stored on master. A previous PR (https://github.com/trynthink/scout/pull/440) added all integration test results to master. This PR provides a way of evaluating the differences between the working branch and master, which include:
Store plot pdfs as CI artifacts so that they can be visually compared against master
New script compare_results.py to:
Output differences in keys between the branches' agg_results and ecm_results (output to agg_results_key_diffs.csv and ecm_results_key_diffs.csv)
Output percent differences in values between the branches' agg_results and ecm_results, as long as values meet an absolute threshold and the differences meet a percent threshold (output to agg_results_value_diffs.csv and ecm_results_value_diffs.csv)
Output percent differences in values between branches' Summary_Data-MAP.xlsx and Summary_Data-TP.xlsx (output to Summary_Data-MAP_percent_diffs.csv and Summary_Data-TP_percent_diffs.csv)
Update the Github Actions workflow so that when there are differences between the branch and master agg_results.json or ecm_results.json, then:
Commit new results and plots (same as before)
Pull down agg_results.json, ecm_results.json, Summary_Data-TP.xlsx, Summary_Data-MAP.xlsx from master, store in tests/integration_tests/results_base
Run tests/integration_tests/compare_results.py
Store the output csvs described above as CI artifacts
Fixes https://github.com/trynthink/scout/issues/415
Introduces a class to compare integration test results on a branch with the results stored on master. A previous PR (https://github.com/trynthink/scout/pull/440) added all integration test results to master. This PR provides a way of evaluating the differences between the working branch and master, which include:
compare_results.py
to:agg_results_key_diffs.csv
andecm_results_key_diffs.csv
)agg_results_value_diffs.csv
andecm_results_value_diffs.csv
)Summary_Data-MAP.xlsx
andSummary_Data-TP.xlsx
(output toSummary_Data-MAP_percent_diffs.csv
andSummary_Data-TP_percent_diffs.csv
)agg_results.json
orecm_results.json
, then:tests/integration_tests/results_base
tests/integration_tests/compare_results.py
Example Outputs Example CI artifacts are found at https://github.com/trynthink/scout/actions/runs/11943859504
Example
*_results_key_diffs.csv
:Example
*_results_value_diffs.csv
:Example
Summary_Data-*_percent_diffs.xlsx
: Same format as original xlsx files, but values are the percent differences