Genentech / jmpost

https://genentech.github.io/jmpost/
17 stars 4 forks source link

Support initial values for constrained parameters #372

Closed gowerc closed 4 months ago

gowerc commented 4 months ago

Closes #268

Generally tried to shield the user from this as much as possible and pushed all code to the behind the scenes. Each model (e.g. LongitudinalGSF) is responsible for setting the constraints on the priors which is done via set_limits().

initialValues() then attempts to draw 100 values and discards any that do not meet the constraint. If there is more than 1 value that meets the constraint then 1 of these is sampled at random.

For random effects parameters that were having their distribution set based on the initial value of the parent I've tried to reduce the variance of the values being produced by implementing a median function. In short this just generates 250 values following the above described method and then just takes the median across these. Originally we were just taking the centre point of the distribution but as users can now specify distributions whose centre point is potentially not a valid value this alternative was required.

Misc changes

github-actions[bot] commented 4 months ago

badge

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      167       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          34       4  88.24%   52, 396, 416, 471
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       126      10  92.06%   146-151, 201, 205, 247, 293, 299
R/JointModelSamples.R                 66       0  100.00%
R/link_generics.R                     13       3  76.92%   59, 76, 91
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalClaretBruno.R           88       6  93.18%   164-168, 183
R/LongitudinalGSF.R                   89       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           44       5  88.64%   102-106
R/LongitudinalSteinFojo.R             80       5  93.75%   155-159
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            254       6  97.64%   507, 651-665
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalClaretBruno.R        52       0  100.00%
R/SimLongitudinalGSF.R                60       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          50       0  100.00%
R/SimSurvival.R                      104       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 11       0  100.00%
R/utilities.R                        144       2  98.61%   13, 335
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               3009     120  96.01%

Diff against main

Filename                       Stmts    Miss  Cover
---------------------------  -------  ------  --------
R/generics.R                      +1       0  +0.36%
R/LongitudinalClaretBruno.R       +5       0  +0.41%
R/LongitudinalGSF.R               +5       0  +100.00%
R/LongitudinalRandomSlope.R       +2       0  +0.54%
R/LongitudinalSteinFojo.R         +4       0  +0.33%
R/Prior.R                        +18      -2  +1.03%
TOTAL                            +35      -2  +0.11%

Results for commit: a0a8a6f923382015dd019b1a77ec3f93d03a50d7

Minimum allowed coverage is 80%

:recycle: This comment has been updated with latest results

github-actions[bot] commented 4 months ago

Unit Tests Summary

    1 files    160 suites   6m 26s :stopwatch:   139 tests   136 :white_check_mark: 3 :zzz: 0 :x: 1 052 runs  1 048 :white_check_mark: 4 :zzz: 0 :x:

Results for commit a9c9f1ab.

:recycle: This comment has been updated with latest results.

github-actions[bot] commented 4 months ago

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
JointModelSamples 💔 $30.05$ $+2.01$ $0$ $0$ $0$ $0$
extract_quantities 💚 $7.52$ $-2.56$ $0$ $0$ $0$ $0$
misc_models 💔 $0.84$ $+4.40$ $0$ $0$ $0$ $0$
model_multi_chain 💚 $4.54$ $-1.41$ $0$ $0$ $0$ $0$
Additional test case details | Test Suite | $Status$ | Time on `main` | $±Time$ | Test Case | |:-----|:----:|:----:|:----:|:-----| | Grid | 💔 | $2.51$ | $+52.53$ | Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser | | GridPrediction | 💔 | $117.08$ | $+2.72$ | GridPrediction_works_as_expected_for_Survival_models | | JointModelSamples | 💔 | $30.05$ | $+2.01$ | print_works_as_expected_for_JointModelSamples | | LongitudinalClaretBruno | 👶 | | $+0.01$ | Can_generate_valid_initial_values | | LongitudinalClaretBruno | 💔 | $4.38$ | $+1.50$ | LongitudinalClaretBruno_works_as_expected_with_default_arguments | | LongitudinalGSF | 👶 | | $+0.01$ | Can_generate_valid_initial_values | | LongitudinalQuantiles | 💚 | $45.99$ | $-42.58$ | Test_that_LongitudinalQuantities_works_as_expected | | LongitudinalRandomSlope | 👶 | | $+0.00$ | Can_generate_valid_initial_values | | LongitudinalSteinFojo | 👶 | | $+0.01$ | Can_generate_valid_initial_values | | LongitudinalSteinFojo | 💔 | $3.95$ | $+1.15$ | LongitudinalSteinFojo_works_as_expected_with_default_arguments | | Prior | 👶 | | $+0.00$ | Limits_work_as_expected | | Prior | 👶 | | $+0.00$ | median_Prior_works_as_expected | | SurvivalExponential | 💔 | $22.29$ | $+2.27$ | SurvivalExponential_can_recover_true_parameter_including_covariates_ | | SurvivalLoglogistic | 💚 | $22.88$ | $-8.48$ | sim_os_loglogistic_is_consistant_with_flexsurv | | SurvivalQuantities | 💚 | $3.49$ | $-3.23$ | SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected | | brierScore | 💔 | $26.46$ | $+10.16$ | brierScore_SurvivalQuantities_returns_same_results_as_survreg | | extract_quantities | 💚 | $7.52$ | $-2.56$ | extract_quantities_works_as_expected | | misc_models | 💔 | $0.84$ | $+4.40$ | Longitudinal_Model_doesn_t_print_sampler_rejection_messages | | model_multi_chain | 💚 | $4.54$ | $-1.41$ | Can_recover_known_distribution_parameters_from_random_slope_model_when_using_multiple_chains | | stan_functions | 💔 | $26.18$ | $+1.35$ | GSF_SLD_function_works_as_expected |

Results for commit 39f98f9d0aea465204c30d16b05357d316fc90bf

♻️ This comment has been updated with latest results.