Implement model validation

[codeanticode]

The prognostic model presented in the 2018 TLID paper is a very simply logistic regression model with the following predictors:

Intercept = -8.540674959 Age = 0.043498988 Severe CNS = 1.011961038 Bleeding = 0.898388975 Jaundice = 2.029453703 AST = 0.002990266 Cr = 0.146072204 K = 0.923083855

We need to apply this model on the new dataset for independent validation.

[KelseyB-code]

See "model_validation.Rmd"

[codeanticode]

We need to add intercept, use at presentation symptoms, and then calculate AUC, calibration curve, sensitivity and specificity.

[KelseyB-code]

See new file "model_validation_symptoms_at_presentation"

[codeanticode]

The performance of the model on the new data is very poor, with sensitivity almost zero, so model is predicting most patients to survive. Some things to look at:

• Debug de code, for example calculate a few predictions by hand and compare with the values from the model
• The descriptive statistics of the continuous variables seem consistent between the old and new data, so do not think that's the problem here
• The binary variables (signs & symptoms and presentation) seem different quite different, with bleeding and CNS at presentation having a very low prevalence in the data. Jaundice had low prevalence in the old data, but is never observed in the new one. So maybe is worth comparing those were calculated, since both bleeding and cns were constructed from other variables. @codeanticode will revise how those variables were constructed in the previous analysis
• Compare performance of model trained on old data that only incorporates continuous variables, since those are arguably more objective @KelseyB-code will run the old lab models on the new data