Genentech / jmpost

https://genentech.github.io/jmpost/
17 stars 4 forks source link

Fix SF Quantities #370

Closed gowerc closed 4 months ago

gowerc commented 4 months ago

Closes #368

Ok so the actual bug fix was trivial was just a case of fixing a simple typo e.g. gsf -> sf. What was a bit more involved though was updating the unit tests to actually catch this error. It would have been easy to just compile all of the models but this would have added a ton of run time especially if we want to test each permutation of model / quantity generation.

Instead I decided to abstract the code that generates the StanModule for the quantities into its own function so that we can just grab that object before its actually compiled. I then put together a mock JointModelSamples object to which I then use to generate the corresponding StanModule and then run the stan parser to make sure theres no obviously variable errors.

I then added the unit test to our other longitudinal models to make sure no other bugs like this exist.

A bit of a red herring in these changes was that my original implementation tried to expand on the write_stan() function. I adapted the function to take a connection stream to which I was going to pipe the output to a variable. I abandoned this implementation for testing this PR but I thought having the function take a stream was a nice change so left it in as I had already written the code (hope thats ok).

gowerc commented 4 months ago

@gravesti - The "files changed" tab on GitHub here is miss-leading; as I've moved a chunk of code (without modification) between 2 functions the diff is just showing the difference between the code chunks immediately before the chunk I moved. I would potentially recommend viewing the complete file and not just the diff.

github-actions[bot] commented 4 months ago

badge

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      167       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          33       4  87.88%   52, 396, 416, 471
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       126      10  92.06%   146-151, 201, 205, 247, 293, 299
R/JointModelSamples.R                 66       0  100.00%
R/link_generics.R                     13       3  76.92%   59, 76, 91
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalClaretBruno.R           83       6  92.77%   156-160, 175
R/LongitudinalGSF.R                   83       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           42       5  88.10%   98-102
R/LongitudinalSteinFojo.R             76       5  93.42%   149-153
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            236       8  96.61%   481, 577, 589-607
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalClaretBruno.R        52       0  100.00%
R/SimLongitudinalGSF.R                53       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          50       0  100.00%
R/SimSurvival.R                      104       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 11       0  100.00%
R/utilities.R                        144       2  98.61%   13, 335
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               2966     122  95.89%

Diff against main

Filename                     Stmts    Miss  Cover
-------------------------  -------  ------  --------
R/JointModel.R                  +1      +2  -1.54%
R/JointModelSamples.R           +3       0  +100.00%
R/LongitudinalSteinFojo.R        0      -1  +1.32%
R/utilities.R                   +1      +1  -0.69%
TOTAL                           +5      +2  -0.06%

Results for commit: 627b514ec361f2a09bbc43009946ac8da7b0c1fb

Minimum allowed coverage is 80%

:recycle: This comment has been updated with latest results

gowerc commented 4 months ago

Updated to add the same test for the new claretBruno model which was merged in this morning

github-actions[bot] commented 4 months ago

Unit Tests Summary

    1 files    154 suites   6m 44s :stopwatch:   136 tests   133 :white_check_mark: 3 :zzz: 0 :x: 1 021 runs  1 017 :white_check_mark: 4 :zzz: 0 :x:

Results for commit 627b514e.

:recycle: This comment has been updated with latest results.

github-actions[bot] commented 4 months ago

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
JointModelSamples 💚 $32.29$ $-1.47$ $0$ $0$ $0$ $0$
compile 💚 $6.56$ $-5.45$ $0$ $0$ $0$ $0$
extract_quantities 💔 $0.87$ $+2.38$ $0$ $0$ $0$ $0$
model_multi_chain 💚 $7.88$ $-2.29$ $0$ $0$ $0$ $0$
Additional test case details | Test Suite | $Status$ | Time on `main` | $±Time$ | Test Case | |:-----|:----:|:----:|:----:|:-----| | Grid | 💚 | $17.25$ | $-4.32$ | Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser | | GridPrediction | 💚 | $138.55$ | $-4.87$ | GridPrediction_works_as_expected_for_Survival_models | | JointModelSamples | 💚 | $32.29$ | $-1.47$ | print_works_as_expected_for_JointModelSamples | | LongitudinalClaretBruno | 💔 | $3.24$ | $+1.19$ | LongitudinalClaretBruno_works_as_expected_with_default_arguments | | LongitudinalClaretBruno | 👶 | | $+0.01$ | Quantity_models_pass_the_parser | | LongitudinalGSF | 💔 | $2.12$ | $+1.47$ | LongitudinalGSF_works_as_expected_with_default_arguments | | LongitudinalGSF | 👶 | | $+0.02$ | Quantity_models_pass_the_parser | | LongitudinalQuantiles | 💚 | $49.51$ | $-1.97$ | Test_that_LongitudinalQuantities_works_as_expected | | LongitudinalRandomSlope | 💔 | $15.26$ | $+3.74$ | Print_method_for_LongitudinalRandomSlope_works_as_expected | | LongitudinalRandomSlope | 👶 | | $+0.00$ | Quantity_models_pass_the_parser | | LongitudinalSteinFojo | 👶 | | $+0.03$ | Quantity_models_pass_the_parser | | SurvivalLoglogistic | 💚 | $9.53$ | $-4.49$ | sim_os_loglogistic_is_consistant_with_flexsurv | | SurvivalQuantities | 💔 | $24.48$ | $+5.64$ | SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected | | brierScore | 💔 | $3.84$ | $+1.83$ | brierScore_SurvivalQuantities_returns_same_results_as_survreg | | compile | 💚 | $6.56$ | $-5.45$ | compileStanModel_doesn_t_error_if_the_directory_doesn_t_exist | | extract_quantities | 💔 | $0.87$ | $+2.38$ | extract_quantities_works_as_expected | | model_multi_chain | 💚 | $7.88$ | $-2.29$ | Can_recover_known_distribution_parameters_from_random_slope_model_when_using_multiple_chains | | stan_functions | 💔 | $25.55$ | $+1.17$ | GSF_SLD_function_works_as_expected |

Results for commit 85811918418fc4f7dcb3472fac71689705f5fb6f

♻️ This comment has been updated with latest results.