search

Genentech / jmpost

https://genentech.github.io/jmpost/
17 stars 4 forks source link

Individual Loo for submodels #408

Closed gowerc closed 2 months ago

gowerc commented 3 months ago

Closes #402

github-actions[bot] commented 3 months ago

badge

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      167       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          34       4  88.24%   52, 396, 416, 471
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       129      10  92.25%   148-153, 203, 207, 249, 295, 301
R/JointModelSamples.R                 66       0  100.00%
R/link_generics.R                     16       4  75.00%   59, 76, 91, 106
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalClaretBruno.R           90       6  93.33%   168-172, 187
R/LongitudinalGSF.R                   95       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           44       5  88.64%   102-106
R/LongitudinalSteinFojo.R             86       5  94.19%   159-163
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            254       6  97.64%   507, 651-665
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalClaretBruno.R        74       0  100.00%
R/SimLongitudinalGSF.R                80       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          69       0  100.00%
R/SimSurvival.R                      116       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalGamma.R                     13       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 13       0  100.00%
R/utilities.R                        144       2  98.61%   13, 335
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               3117     121  96.12%

Diff against main

Filename          Stmts    Miss  Cover
--------------  -------  ------  -------
R/JointModel.R       +3       0  +0.18%
TOTAL                +3       0  +0.00%

Results for commit: bb458ad8dbdc16683ae219e39f6f25edd5404b88

Minimum allowed coverage is 80%

:recycle: This comment has been updated with latest results

github-actions[bot] commented 3 months ago

Unit Tests Summary

    1 files    178 suites   6m 9s :stopwatch:   156 tests   144 :white_check_mark: 12 :zzz: 0 :x: 1 091 runs  1 078 :white_check_mark: 13 :zzz: 0 :x:

Results for commit bb458ad8.

:recycle: This comment has been updated with latest results.

github-actions[bot] commented 3 months ago

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
JointModelSamples 💔 $31.39$ $+1.20$ $0$ $0$ $0$ $0$
brms_compare 👶 $+0.02$ $+1$ $+1$ $0$ $0$
compile 💔 $0.77$ $+2.09$ $0$ $0$ $0$ $0$
extract_quantities 💚 $4.55$ $-1.72$ $0$ $0$ $0$ $0$
misc_models 💔 $4.69$ $+1.39$ $0$ $0$ $0$ $0$
model_multi_chain 💚 $3.13$ $-2.05$ $0$ $0$ $0$ $0$
Additional test case details | Test Suite | $Status$ | Time on `main` | $±Time$ | Test Case | |:-----|:----:|:----:|:----:|:-----| | Grid | 💚 | $52.46$ | $-2.25$ | Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser | | GridPrediction | 💔 | $123.11$ | $+2.76$ | GridPrediction_works_as_expected_for_Survival_models | | JointModel | 👶 | | $+0.01$ | Log_Lik_variables_are_produced_correctly | | JointModelSamples | 💔 | $31.39$ | $+1.20$ | print_works_as_expected_for_JointModelSamples | | LongitudinalQuantiles | 💔 | $3.48$ | $+2.56$ | Test_that_LongitudinalQuantities_works_as_expected | | SurvivalQuantities | 💚 | $5.78$ | $-1.80$ | SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected | | brms_compare | 👶 | | $+0.07$ | jmpost_and_brms_get_similar_loo_for_longitudinal_models | | brms_compare | 👶 | | $+0.02$ | jmpost_and_brms_get_similar_loo_for_survival_models | | compile | 💔 | $0.77$ | $+2.09$ | compileStanModel_doesn_t_error_if_the_directory_doesn_t_exist | | extract_quantities | 💚 | $4.55$ | $-1.72$ | extract_quantities_works_as_expected | | misc_models | 💔 | $4.69$ | $+1.39$ | Longitudinal_Model_doesn_t_print_sampler_rejection_messages | | model_multi_chain | 💚 | $3.13$ | $-2.05$ | Can_recover_known_distribution_parameters_from_random_slope_model_when_using_multiple_chains | | stan_functions | 💚 | $29.17$ | $-1.11$ | GSF_SLD_function_works_as_expected |

Results for commit 68276611eac55324dfd634b53dbfb6133a6d3443

♻️ This comment has been updated with latest results.

gowerc commented 2 months ago

Actually before merging I think it might be worth adding a few CRON tests that just compare the loo values to that of brms to ensure consistency.

gowerc commented 2 months ago

@gravesti - addressed your comment :) - Also added an URL field to the description file which was throwing up issues for some reason now (probs a package update that I didn't previously have)

gravesti commented 2 months ago

@gowerc I think you might not have pushed your latest changes yet :)

gowerc commented 2 months ago

@gowerc I think you might not have pushed your latest changes yet :)

I think I might have just added that comment on the wrong PR, looking over it I've confused myself now. But the "URL" change looks like it was introduced in the Gamma model PR so maybe I made the change on the wrong branch...

gowerc commented 2 months ago

@gravesti - Looks like merging in main dismissed your approval sorry.