Genentech / jmpost

https://genentech.github.io/jmpost/
17 stars 4 forks source link

Add toggle for Graph Snapshot Testing #361

Closed gowerc closed 5 months ago

gowerc commented 5 months ago

No corresponding issue.

Unit tests were failing on internal CICD build due to snapshot discrepancies, as far as I can tell this is minor white space stuff on graphs due to differences in package versions (I've posted the diff below). To address this I've added a toggle for graph snapshots which is enabled for the GitHub CICD but disabled for the in house builds.

I opted for a wrapper around the expect* function rather than skipping the test as this way it will still through an error on the internal build if the plot generation code fails its only the final comparison that is suppressed.

> diff  \
    survival-plot-no-wrap-no-ci-km-ggplot2.new.svg  \
    survival-plot-no-wrap-no-ci-km-ggplot2.svg

21c21
< <rect x='0.00' y='0.00' width='720.00' height='576.00' style='stroke-width: 1.07; stroke: #FFFFFF; fill: #FFFFFF;' />
---
> <rect x='0.000000000000064' y='0.00' width='720.00' height='576.00' style='stroke-width: 1.07; stroke: #FFFFFF; fill: #FFFFFF;' />
github-actions[bot] commented 5 months ago

badge

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      166       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          31       4  87.10%   51, 395, 414, 449
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       123       8  93.50%   141-143, 193, 197, 239, 285, 291
R/JointModelSamples.R                 59       0  100.00%
R/link_generics.R                     13       3  76.92%   59, 76, 91
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalGSF.R                   82       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           41       5  87.80%   93-97
R/LongitudinalSteinFojo.R             75       5  93.33%   143-147
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            236       8  96.61%   481, 577, 589-607
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalGSF.R                53       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          50       0  100.00%
R/SimSurvival.R                      104       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 11       0  100.00%
R/utilities.R                        143       1  99.30%   13
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               2814     113  95.98%

Diff against main

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: a36dbdca071488bcf902e8959d2e0da17888e4bd

Minimum allowed coverage is 80%

:recycle: This comment has been updated with latest results

github-actions[bot] commented 5 months ago

Unit Tests Summary

  1 files  140 suites   7m 54s :stopwatch: 131 tests 128 :white_check_mark: 3 :zzz: 0 :x: 976 runs  973 :white_check_mark: 3 :zzz: 0 :x:

Results for commit a36dbdca.

:recycle: This comment has been updated with latest results.

github-actions[bot] commented 5 months ago

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
compile 💚 $3.12$ $-1.79$ $0$ $0$ $0$ $0$
extract_quantities 💔 $9.87$ $+1.86$ $0$ $0$ $0$ $0$
model_multi_chain 💚 $35.39$ $-2.07$ $0$ $0$ $0$ $0$
Additional test case details | Test Suite | $Status$ | Time on `main` | $±Time$ | Test Case | |:-----|:----:|:----:|:----:|:-----| | Grid | 💔 | $25.20$ | $+1.45$ | Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser | | GridPrediction | 💔 | $157.23$ | $+1.73$ | GridPrediction_works_as_expected_for_Survival_models | | SurvivalExponential | 💔 | $13.97$ | $+1.54$ | SurvivalExponential_can_recover_true_parameter_including_covariates_ | | SurvivalLoglogistic | 💚 | $17.28$ | $-1.42$ | sim_os_loglogistic_is_consistant_with_flexsurv | | SurvivalQuantities | 💔 | $11.76$ | $+1.88$ | SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected | | brierScore | 💔 | $34.89$ | $+1.57$ | brierScore_SurvivalQuantities_returns_same_results_as_survreg | | compile | 💚 | $3.12$ | $-1.79$ | compileStanModel_doesn_t_error_if_the_directory_doesn_t_exist | | extract_quantities | 💔 | $9.87$ | $+1.86$ | extract_quantities_works_as_expected | | model_multi_chain | 💚 | $35.39$ | $-2.07$ | Can_recover_known_distribution_parameters_from_random_slope_model_when_using_multiple_chains | | stan_functions | 💔 | $16.14$ | $+2.34$ | GSF_SLD_function_works_as_expected |

Results for commit 14a09bddf4142ccecf0d43704158834aeb432aa0

♻️ This comment has been updated with latest results.