Closed mgramigna closed 1 year ago
St.:grey_question: |
Category | Percentage | Covered / Total |
---|---|---|---|
🟢 | Statements | 85.39% | 2023/2369 |
🟡 | Branches | 73.97% | 1802/2436 |
🟢 | Functions | 87.5% | 357/408 |
🟢 | Lines | 85.69% | 1952/2278 |
379 tests passing in 29 suites.
Report generated by 🧪jest coverage report action from 0d309b7179ef7f0079b89b7d2856b4e927635ffd
Summary
Adds scripts for comparing calculation results across
ecqm-content
repos. See additions tocontributing.md
for details on what exactly is happening.New behavior
None in source code, new script exists.
Code changes
Le script. Update docs
Testing guidance
I've simulated an error on the
simulated-regression-error
branch, where for a given calculation, there is a 25% chance that it will incorrectly set numer/denom to true when IPP is false. If you run the script and get no failures at all, go buy a lottery ticket! To test:1)
git checkout simulated-regression-error
2) Run the following:This will run regression with this PR's branch as the base, and the error branch as the "one under test." Conceptually this is simulating a scenario where you might be reviewing the code on
simulated-regression-error
and want to make sure that it didn't change any functionality from its base branch (in this caseadd-regression-script
). In general the base branch will be master, but right now it won't work againstmaster
since the regression script isn't checked in there yet.You should get some errors (this won't be idempotent since errors are random):
This indicates that calculation results differ across the branches
-v
during the regression run which will print the diffs of the actual results (NOTE: the errors you get will be different than step 2 since it's random):Diffs will be printed to the terminal in realtime: