Claret Bruno Model

[gowerc]

Closes #338

Most details are in the linked issue. Otherwise not too much special here just based most off the code off of the existing SF model with no real fundamental changes.

Only concern I have is that the ESS seems to be quite low at ~10%. Though this model has a double exponential and some terms that are hard to pick a part so I don't think its a validity concern, its not like the old non-central b parameter that only had single figure ess even with thousands of samples.... but yer not 100% sure that this is ok to ignore but my gut feeling says it is.

EDIT - Talking to Francois he was saying that the SB model tends to have slightly worse performance metrics than the GSF which means we are roughly in line with this expectation so I think this is safe to ignore.

[github-actions[bot]]

Unit Tests Summary

    1 files    150 suites   6m 46s :stopwatch:   135 tests   132 :white_check_mark: 3 :zzz: 0 :x: 1 013 runs  1 009 :white_check_mark: 4 :zzz: 0 :x:

Results for commit e667dcda.

:recycle: This comment has been updated with latest results.

[github-actions[bot]]

Unit Test Performance Difference

Test Suite	$Status$	Time on main	$±Time$	$±Tests$	$±Skipped$	$±Failures$	$±Errors$
LongitudinalClaretBruno	👶		$+0.00$	$+1$	$+1$	$0$	$0$
SimLongitudinalClaretBruno	👶		$+0.00$	$+1$	$0$	$0$	$0$

Additional test case details

| Test Suite | $Status$ | Time on `main` | $±Time$ | Test Case | |:-----|:----:|:----:|:----:|:-----| | Grid | 💚 | $23.17$ | $-3.59$ | Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser | | GridPrediction | 💔 | $130.31$ | $+3.63$ | GridPrediction_works_as_expected_for_Survival_models | | LongitudinalClaretBruno | 👶 | | $+0.00$ | Can_recover_known_distributional_parameters_from_a_SF_joint_model | | LongitudinalClaretBruno | 👶 | | $+0.01$ | Centralised_parameterisation_compiles_without_issues | | LongitudinalClaretBruno | 👶 | | $+3.13$ | LongitudinalClaretBruno_works_as_expected_with_default_arguments | | LongitudinalClaretBruno | 👶 | | $+0.01$ | Non_Centralised_parameterisation_compiles_without_issues | | LongitudinalClaretBruno | 👶 | | $+0.00$ | Print_method_for_LongitudinalClaretBruno_works_as_expected | | LongitudinalRandomSlope | 💔 | $12.22$ | $+2.72$ | Print_method_for_LongitudinalRandomSlope_works_as_expected | | SimLongitudinalClaretBruno | 👶 | | $+0.14$ | SimLongitudinalClaretBruno_works_as_expected | | SimLongitudinalClaretBruno | 👶 | | $+0.00$ | print_methods_work_as_expected | | SurvivalLoglogistic | 💚 | $12.07$ | $-2.26$ | sim_os_loglogistic_is_consistant_with_flexsurv | | SurvivalQuantities | 💔 | $20.77$ | $+2.59$ | SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected | | brierScore | 💚 | $7.35$ | $-2.32$ | brierScore_SurvivalQuantities_returns_same_results_as_survreg | | brierScore | 👶 | | $+0.00$ | match_order_works_as_expected |

Results for commit dc4538b6fc2773813686a60dc55efcd564a51ca5

♻️ This comment has been updated with latest results.

[gowerc]

Hmm @danielinteractive , would you mind focusing on the dsld code. For some reason the joint model is working perfectly with no links but as soon as I add the DSLD link it and the lambda_0 parameter go very wrong.

For example with link_dsld = -0.1, link_ttg=0.2 and lambda_0=1 I am getting the following:

# A tibble: 3 × 7
  variable         mean     q01     q99  rhat ess_bulk ess_tail
  <chr>           <dbl>   <dbl>   <dbl> <dbl>    <dbl>    <dbl>
1 link_dsld     -0.0418 -0.0454 -0.0384  1.00    2128.    1651.
2 link_ttg       0.216  -0.917   1.35    1.00    3398.    1370.
3 sm_exp_lambda  4.75    3.27    6.51    1.00    2905.    1726.

I've been staring at the code for ages now and I can't see any obvious place where I have gone wrong :(

EDIT - NVM just spotted a couple of mistakes / missing brackets in my formula. Am re-testing again now... EDIT2 - ok still getting incorrect results after applying the fix :(

  variable         mean     q01     q99  rhat ess_bulk ess_tail
  <chr>           <dbl>   <dbl>   <dbl> <dbl>    <dbl>    <dbl>
1 link_dsld     -0.0358 -0.0394 -0.0324 1.00     2181.    1508.
2 link_ttg       0.215  -0.930   1.37   1.00     3030.    1359.
3 sm_exp_lambda  8.36    5.86   11.5    0.999    3197.    1493.

[github-actions[bot]]

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      167       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          33       4  87.88%   51, 395, 415, 470
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       125       8  93.60%   146-148, 198, 202, 244, 290, 296
R/JointModelSamples.R                 63       0  100.00%
R/link_generics.R                     13       3  76.92%   59, 76, 91
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalClaretBruno.R           83       6  92.77%   156-160, 175
R/LongitudinalGSF.R                   83       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           42       5  88.10%   98-102
R/LongitudinalSteinFojo.R             76       6  92.11%   117, 149-153
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            236       8  96.61%   481, 577, 589-607
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalClaretBruno.R        52       0  100.00%
R/SimLongitudinalGSF.R                53       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          50       0  100.00%
R/SimSurvival.R                      104       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 11       0  100.00%
R/utilities.R                        143       1  99.30%   13
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               2961     120  95.95%

Diff against main

Filename                          Stmts    Miss  Cover
------------------------------  -------  ------  --------
R/brier_score.R                      +1       0  +100.00%
R/LongitudinalClaretBruno.R         +83      +6  +92.77%
R/SimLongitudinalClaretBruno.R      +52       0  +100.00%
TOTAL                              +136      +6  -0.02%

Results for commit: e667dcdafd7fddfd4ba99007f2059b034909e4a8

Minimum allowed coverage is 80%

:recycle: This comment has been updated with latest results

[gowerc]

@danielinteractive - Unfortunately still not working :(

With real values of link_dsld = -0.1 link_ttg = 0.2 lambda = 1

I'm getting samples of:

  variable         mean     q01     q99  rhat ess_bulk ess_tail
  <chr>           <dbl>   <dbl>   <dbl> <dbl>    <dbl>    <dbl>
1 link_dsld     -0.0390 -0.0433 -0.0345 1.00     3397.    1734.
2 link_ttg       0.189  -0.889   1.28   0.999    4259.    1487.
3 sm_exp_lambda  5.62    3.92    7.68   1.00     3781.    1384

[gowerc]

@danielinteractive - @gravesti managed to work out that there was no issue with the implementation, the problem was that my simulation parameters were too extreme and the majority of people were dying instantly. Using more sensible parameters the model works as expected and is able to recover all the known parameters 🎉

[danielinteractive]

Great news!! Thanks both!!

[gowerc]

@danielinteractive are you able to still do the review for this or should I transfer to @gravesti instead?

[danielinteractive]

Hi @gowerc , yeah sure, anything specific to check still?

[gowerc]

@danielinteractive - Not really, I would say the Stan code is probs the highest risk area though.