improve chronic disease transition models

[ld-archer]

Using the T-tests as a benchmark for improvement, we need to improve the transition models for some chronic diseases.

We will focus on Cancer, Diabetes, Heart Disease, and Hypertension first, as these are some of the most important chronic diseases, and some are highly influential in other chronic diseases.

Here are the T-tests as a starting point:

variable	fem_mean_wave3	elsa_mean_wave3	p_value_wave3	fem_mean_wave4	elsa_mean_wave4	p_value_wave4	fem_mean_wave5	elsa_mean_wave5	p_value_wave5	fem_mean_wave6	elsa_mean_wave6	p_value_wave6	fem_mean_wave7	elsa_mean_wave7	p_value_wave7	fem_mean_wave8	elsa_mean_wave8	p_value_wave8
Any ADLs	0.2034	0.21265	0.19503	0.20566	0.22858	0.00341	0.21061	0.23353	0.005	0.2248	0.23014	0.53101	0.24046	0.21925	0.01868	0.25074	0.22498	0.00834
Any IADLs	0.22006	0.22211	0.77756	0.22696	0.24747	0.01084	0.23492	0.2548	0.01807	0.24922	0.26006	0.22174	0.26668	0.25179	0.11507	0.28758	0.26585	0.03525
Cancer ever	0.10529	0.08634	0.00013	0.12412	0.0972	0	0.14332	0.11917	0.00013	0.16537	0.12381	0	0.18708	0.1466	0	0.20995	0.16397	0
Diabetes ever	0.1134	0.0937	0.00012	0.12937	0.11321	0.00658	0.14451	0.12803	0.01107	0.15961	0.13755	0.00165	0.17455	0.15406	0.00933	0.18565	0.15097	4E-05
Heart disease ever	0.22072	0.17494	0	0.24355	0.1983	0	0.26794	0.22183	0	0.29371	0.22953	0	0.32054	0.25602	0	0.34867	0.28985	0
Hypertension ever	0.48354	0.4483	5E-05	0.51649	0.48162	0.00019	0.54569	0.50639	5E-05	0.57324	0.50793	0	0.59931	0.53135	0	0.62452	0.5384	0
Lung disease ever	0.08997	0.07187	7E-05	0.09918	0.07859	5E-05	0.10748	0.08631	0.00011	0.1182	0.08619	0	0.12758	0.09168	0	0.13645	0.08975	0
Stroke ever	0.05877	0.05037	0.02895	0.06486	0.06373	0.8043	0.07241	0.06743	0.30549	0.07907	0.07311	0.25971	0.08516	0.0833	0.75748	0.09339	0.08899	0.50812

Ideas: So far, I have made some reasonable improvements just by tweaking the transition models. Improving the logbmi model made probably the biggest difference of anything for a couple of variables. I will continue to do this, but also look into some other variables from ELSA we can include. Examples being high cholesterol and hip fracture as 2 variables that have made a noticeable improvement.

This issue continues the work started in #41, #45, #48 & #49.

[ld-archer]

Cancer: Cancer rates from the model are much higher than ELSA by wave 8 (20% vs 16%). To improve the cancer model:

• [x] Improve smoke_start and smoke_stop models
  • Smoking models weren't terrible to start, but stripping them back to the minimal and building them iteratively has improved them slightly. Not made a massive difference to the chronic diseases though.
• [ ] Improve drinking model
  • [ ] Remove drinkd model
• [x] Include marital status (l2married)
  • Not much effect
• [ ] Improve psyche model? Worth a try

[ld-archer]

Update

Made big improvements in the chronic disease models, and risk behaviours look slightly better (both still WIP).

variable	fem_mean_wave3	elsa_mean_wave3	p_value_wave3	fem_mean_wave4	elsa_mean_wave4	p_value_wave4	fem_mean_wave5	elsa_mean_wave5	p_value_wave5	fem_mean_wave6	elsa_mean_wave6	p_value_wave6	fem_mean_wave7	elsa_mean_wave7	p_value_wave7	fem_mean_wave8	elsa_mean_wave8	p_value_wave8
Any ADLs	0.20446	0.21265	0.25128	0.20819	0.22858	0.00916	0.21407	0.23353	0.01708	0.22762	0.23014	0.76752	0.2423	0.21925	0.01056	0.25394	0.22498	0.003
Any IADLs	0.22208	0.22211	0.99636	0.23023	0.24747	0.03216	0.238	0.2548	0.04566	0.25351	0.26006	0.46049	0.27101	0.25179	0.04184	0.29071	0.26585	0.01591
Cancer ever	0.10008	0.08634	0.00535	0.11492	0.0972	0.00144	0.12968	0.11917	0.094	0.14571	0.12381	0.00108	0.16072	0.1466	0.06695	0.17607	0.16397	0.16124
Diabetes ever	0.11092	0.0937	0.00077	0.12531	0.11321	0.04139	0.13843	0.12803	0.10811	0.15527	0.13755	0.01134	0.16736	0.15406	0.09058	0.17651	0.15097	0.00229
Heart disease ever	0.21407	0.17494	0	0.23299	0.1983	0	0.25353	0.22183	8E-05	0.27484	0.22953	0	0.2967	0.25602	2E-05	0.31941	0.28985	0.00524
Hypertension ever	0.47905	0.4483	0.0004	0.50866	0.48162	0.00377	0.53503	0.50639	0.00303	0.55888	0.50793	0	0.58066	0.53135	1E-05	0.602	0.5384	0
Lung disease ever	0.09032	0.07187	5E-05	0.09938	0.07859	4E-05	0.10795	0.08631	8E-05	0.11846	0.08619	0	0.12791	0.09168	0	0.13641	0.08975	0
Psychological problems ever	0.09947	0.08478	0.00268	0.11003	0.09286	0.00164	0.12086	0.09286	0	0.13097	0.09792	0	0.14043	0.10107	0	0.151	0.11419	0
Stroke ever	0.05912	0.05037	0.02287	0.06475	0.06373	0.82334	0.07223	0.06743	0.32293	0.07805	0.07311	0.3497	0.08433	0.0833	0.86354	0.09276	0.08899	0.5701

[ld-archer]

Something that would benefit the smoking and drinking models is an indicator of intensity. We've included this before with the smokef (# cigs) and drink_stat vars, but neither of these worked particularly well. Now might be a good time to give that a try, going to spin this comment off into a new issue.

[ld-archer]

Models are now pretty reasonable, and logic to calculate them is in the debug document.