search

pharmpy / pharmpy

A library and toolkit for pharmacometrics
https://pharmpy.github.io
GNU General Public License v3.0
48 stars 14 forks source link

dOFV cutoff setting in model search #337

Open rosemarychen99 opened 3 years ago

rosemarychen99 commented 3 years ago

Currently, one can only set one cutoff (e.g. 3.84) for dOFV for model selection in model search algorithm. But the cutoff for likelihood ratio test depends on the difference of parameters. Adding 1 parameter: 3.84; adding 2 parameter: 6 ....

rikardn commented 3 years ago

@rosemarychen99 Do you think we should set the dOFV cutoff dynamically depending on the difference of the number of parameters in the two models?

rosemarychen99 commented 3 years ago

The dOFV cutoff is used for LRT, which is to compare two nested models. The issue is two models are not necessary nested during model development. As a result, it is a complicated issues. As far as I know, Pharmpy cannot detect the two models are nested or not. From a big picture, it is related to what selection criteria we should use during model search. Options are LRT (need cutoff), AIC or BIC (just compare the values), or combined LRT and AIC/BIC (which is the common way during manual search). Regarding AIC and BIC, their formula are a bit different in the case of mixed effect model compared simple regression. We do need more investigation regarding which selection criteria to choose. I guess we can come back to this issue when we have answers to the "bigger" questions.