metrumresearchgroup / bbi

Next generation modeling platform
11 stars 2 forks source link

METHOD=CHAIN and METHOD=BAYES together #288

Closed kylebaron closed 1 year ago

kylebaron commented 1 year ago

Summary

Ref: https://github.com/metrumresearchgroup/bbi/pull/272/files#r965087052

When we run Bayesian estimation, we use two $ESTIMATION records

$EST METHOD = CHAIN ...
$EST METHOD = BAYES ...

The METHOD=CHAIN record does not produce any parameter estimates.

So you will find two estimation methods in the .lst file (when looking for #METH), but there will only be one set of estimates.

This setup currently produces an error:

 - Installed Version: 3.2.0  - Current release: 3.2.0
> x <- read_model(here("model/nonmem/pk/7/7_1"))
> y <- model_summary(x)
Error: nonmem_summary('/data/home/kyleb/project.mrg/bi/dinamo/model/nonmem/pk/7/7_1') failed with the following error. This may be because the modeling run has not finished successfully.

ERROR:
Error in check_status_code(p$get_exit_status(), output, .cmd_args):

`bbi nonmem summary 7_1 --json` returned status code 1 -- STDOUT and STDERR:
{
"run_details": {
"postprocess_time": 0,
"cpu_time": 0
},
"run_heuristics": {
"covariance_step_aborted": false,
"large_condition_number": false,
"eigenvalue_issues": false,
"correlations_not_ok": false,
"parameter_near_boundary": false,
"hessian_reset": false,
"has_final_zero_gradient": false,
"minimization_terminated": false,
"eta_pval_significant": false,
"prderr": false
},
"parameter_names": {},
"success": false,
"error_msg": "lengths of parameter data (1) and estimation methods (2) differ. Incomplete run?"
}
kylebaron commented 1 year ago

From NM-Help; these are the valid values for METHOD in $ESTIM; I believe that METHOD=CHAIN is the only one that doesn't do some estimation.

 METHOD=kind
      Values for kind are:

      0 or ZERO
           Always set etas to 0 during the computation of the objective
           function.  Also called the "first order (FO) method."   This
           is the default.

      1 or CONDITIONAL
           Use  conditional  estimates for the etas during the computa-
           tion of  the  objective  function.   METHOD=1  (without  the
           LAPLACIAN  option)  is  also  called the "first order condi-
           tional estimation (FOCE) method."  The conditional estimates
           of  the  etas are referred to as Conditional Parametric Etas
           (CPE).

           METH=COND NOLAPLACIAN is referred to as the FOCE method.
           METH=COND LAPLACE is referred to as the Laplace method.
           METH=COND NOLAPLACE CENTERING is referred to as the Centering FOCE method.
           METH=COND LAPLACE CENTERING is referred to as the Centering Laplace method.

      HYBRID
           Use conditional estimates for the etas during  the  computa-
           tion  of the objective function, with the exception of those
           etas listed in the ZERO option.  Cannot be used with  LAPLA-
           CIAN or CENTERING.

      The  following  methods  are  new to NONMEM 7.  When any of these
      methods are used, the data are inferred  to  be  population,  and
      METHOD=1  is  supplied  if  it is not already present.  The first
      four methods are referred  to  as  EM  (Expectation-Maximization)
      Methods.

      ITS  Use  the  iterative two-stage method.  This method evaluates
           the conditional mode and first order  approximation  of  the
           conditional  variance  of parameters of individuals by maxi-
           mizing the posterior density.  This integration step is  the
           same  as used in the FOCE method.  Population parameters are

           updated from individuals' conditional  mode  parameters  and
           their approximate variances by single iteration maximization
           steps.

      IMP  Use the Monte-Carlo Importance  Sampling  Expectation  Maxi-
           mization method.  This method evaluates the conditional mean
           and variance of parameters of  individuals  by  Monte  Carlo
           sampling.  It uses the posterior density, which incorporates
           the likelihood of parameters relative  to  population  means
           and variances with the individual's observed data.  The nor-
           mal density near the mean or mode of the posterior  is  used
           as a proposal density, then weighted according to the poste-
           rior density as a correction.

      IMPMAP
           Use the Importance Sampling method assisted by Mode a Poste-
           riori  (MAP)  estimation.   At  each  iteration, conditional
           modes and conditional first order variances are evaluated as
           in  the  ITS or FOCE method.  These are then used as parame-
           ters to the multivariate normal  proposal  density  for  the
           Monte-Carlo importance sampling step.

      SAEM Use  the  Stochastic  Approximation Expectation Maximization
           method.  As in importance sampling, random samples are  gen-
           erated  from normal proposal densities.  However, instead of
           always being centered at the mean or mode of  the  posterior
           density,  the  proposal  density is centered at the previous
           sample position.

      BAYES
           Use the Markov Chain Monte Carlo  (MCMC)  Bayesian  Analysis
           method.  The goal of the MCMC Bayesian analysis is to obtain
           a large sample set of probable population parameters.  Vari-
           ous summary statistics of the population parameters may then
           be obtained such as means and confidence ranges.

      DIRECT
           Requests Monte Carlo Direct  Sampling.   Creates  completely
           independent samples (unlike MCMC), and there is no chance of
           causing bias if the sampling density is not  similar  enough
           to  the  conditional  density  (unlike IMP).  However, it is
           very inefficient, requiring ISAMPLE  settings  of  10000  to
           300000 to properly estimate the problem.

      NUTS (NM74)
           Requests  No U-Turn Sampling (NUTS) Markov Chain Monte Carlo
           (MCMC) Bayesian Analysis Method.   Options  unique  to  this
           method  are  listed  alphabetically  under  NUTS_....  Other
           options of interest with their defaults are as follows:
                MASSRESET=-1
                MADAPT=-1
                KAPPA=1
                TTDF=0
                OLKJDF=0
                OVARF=1
                SLKJDF=0
                SVARF=1

      CHAIN
           Allows the user to create a series of random initial  values
           of  THETAs and OMEGA's, or for reading in initial population
           parameters from a file of rectangular (rows/column)  format.
           Applies only to the Estimation Step.