search

UUPharmacometrics / PsN

Perl-Speaks-NONMEM
https://uupharmacometrics.github.io/PsN
GNU General Public License v2.0
65 stars 19 forks source link

Cannot Summarize NUTS Method #98

Closed billdenney closed 5 years ago

billdenney commented 6 years ago

I have a model with 2 estimation steps: FOCEI followed by NUTS.

$ESTIMATION
  SIGL=9 NSIG=2
  NOABORT
  METHOD=CONDITIONAL INTERACTION
  LAPLACIAN
  NUMERICAL SLOW
  MAXEVAL=9999 PRINT=1
  SADDLE_RESET=1
  FILE=run28foce.ext

$EST METHOD=NUTS
  INTERACTION
  LAPLACIAN
  PRINT=1
  NBURN=200
  NITER=500
  CTYPE=2
  NOPRIOR=0

According to the .lst file, it appeared to complete successfully:


 iteration          500 MCMCOBJ=    1093.58352180651
 BURN-IN WAS COMPLETED

 #TERM:
 STATISTICAL PORTION WAS COMPLETED

 TOTAL DATA POINTS NORMALLY DISTRIBUTED (N):          179
 N*LOG(2PI) CONSTANT TO OBJECTIVE FUNCTION:    328.979994887273
 OBJECTIVE FUNCTION VALUE WITHOUT CONSTANT:    1824.85955048676
 OBJECTIVE FUNCTION VALUE WITH CONSTANT:       2153.83954537403
 REPORTED OBJECTIVE FUNCTION DOES NOT CONTAIN CONSTANT

 TOTAL EFFECTIVE ETAS (NIND*NETA):                           144
 NIND*NETA*LOG(2PI) CONSTANT TO OBJECTIVE FUNCTION:    264.654297562946
 OBJECTIVE FUNCTION VALUE WITHOUT CONSTANT:    1824.85955048676
 OBJECTIVE FUNCTION VALUE WITH CONSTANT:       2089.51384804970
 REPORTED OBJECTIVE FUNCTION DOES NOT CONTAIN CONSTANT

 #TERE:
 Elapsed estimation  time in seconds:   174.76
 Elapsed covariance  time in seconds:     0.00
1

 ************************************************************************************************************************
 ********************                                                                                ********************
 ********************                              NUTS BAYESIAN ANALYSIS                            ********************
 #OBJT:**************                       AVERAGE VALUE OF LIKELIHOOD FUNCTION                     ********************
 ********************                                                                                ********************
 ************************************************************************************************************************

$ sumo modelfit_dir1/NM_run1/psn.lst
Did not find expected information under METH: NUTS Bayesian Analysis number 2

Unable to read everything from outputfile, parser error message:
Did not find expected information under METH: NUTS Bayesian Analysis number 2

The info that shows up under what I think is that section is:

 #TBLN:      2
 #METH: NUTS Bayesian Analysis

 ESTIMATION STEP OMITTED:                 NO
 ANALYSIS TYPE:                           POPULATION
 NUMBER OF SADDLE POINT RESET ITERATIONS:      1
 HESSIAN USED IN SADDLE POINT RESET: VARIABLE METRIC
 GRADIENT METHOD USED:               SLOW
 CONDITIONAL ESTIMATES USED:              YES
 CENTERED ETA:                            NO
 EPS-ETA INTERACTION:                     YES
 LAPLACIAN OBJ. FUNC.:                    NO
 NO. OF FUNCT. EVALS. ALLOWED:            9999
 NO. OF SIG. FIGURES REQUIRED:            2
 INTERMEDIATE PRINTOUT:                   YES
 ESTIMATE OUTPUT TO MSF:                  NO
 ABORT WITH PRED EXIT CODE 1:             NO
 IND. OBJ. FUNC. VALUES SORTED:           NO
 NUMERICAL DERIVATIVE
       FILE REQUEST (NUMDER):               NONE
 MAP (ETAHAT) ESTIMATION METHOD (OPTMAP):   0
 ETA HESSIAN EVALUATION METHOD (ETADER):    0
 INITIAL ETA FOR MAP ESTIMATION (MCETA):    0
 SIGDIGITS FOR MAP ESTIMATION (SIGLO):      9
 GRADIENT SIGDIGITS OF
       FIXED EFFECTS PARAMETERS (SIGL):     9
 NOPRIOR SETTING (NOPRIOR):                 OFF
 NOCOV SETTING (NOCOV):                     OFF
 DERCONT SETTING (DERCONT):                 OFF
 FINAL ETA RE-EVALUATION (FNLETA):          ON
 EXCLUDE NON-INFLUENTIAL (NON-INFL.) ETAS
       IN SHRINKAGE (ETASTYPE):             NO
 NON-INFL. ETA CORRECTION (NONINFETA):      OFF
 RAW OUTPUT FILE (FILE): psn.ext
 EXCLUDE TITLE (NOTITLE):                   NO
 EXCLUDE COLUMN LABELS (NOLABEL):           NO
 FORMAT FOR ADDITIONAL FILES (FORMAT):      S1PE12.5
 PARAMETER ORDER FOR OUTPUTS (ORDER):       TSOL
 WISHART PRIOR DF INTERPRETATION (WISHTYPE):0
 KNUTHSUMOFF:                               0
 INCLUDE LNTWOPI:                           NO
 INCLUDE CONSTANT TERM TO PRIOR (PRIORC):   NO
 INCLUDE CONSTANT TERM TO OMEGA (ETA) (OLNTWOPI):NO
 EM OR BAYESIAN METHOD USED:                MCMC BAYESIAN (BAYES)
 MU MODELING PATTERN (MUM):
 GRADIENT/GIBBS PATTERN (GRD):
 AUTOMATIC SETTING FEATURE (AUTO):          OFF
 CONVERGENCE TYPE (CTYPE):                  2
 KEEP ITERATIONS (THIN):            1
 CONVERGENCE INTERVAL (CINTERVAL):          1
 CONVERGENCE ITERATIONS (CITER):            10
 CONVERGENCE ALPHA ERROR (CALPHA):          5.000000000000000E-02
 BURN-IN ITERATIONS (NBURN):                200
 ITERATIONS (NITER):                        500
 ANEAL SETTING (CONSTRAIN):                 1
 STARTING SEED FOR MC METHODS (SEED):       11456
 MC SAMPLES PER SUBJECT (ISAMPLE):          1
 RANDOM SAMPLING METHOD (RANMETHOD):        3U
 PROPOSAL DENSITY SCALING RANGE
              (ISCALE_MIN, ISCALE_MAX):     1.000000000000000E-06   ,1000000.00000000
 SAMPLE ACCEPTANCE RATE (IACCEPT):          0.400000000000000
 METROPOLIS HASTINGS POPULATION SAMPLING FOR NON-GIBBS
 SAMPLED THETAS AND SIGMAS:
 PROPOSAL DENSITY SCALING RANGE
              (PSCALE_MIN, PSCALE_MAX):   1.000000000000000E-02   ,1000.00000000000
 SAMPLE ACCEPTANCE RATE (PACCEPT):                       0.500000000000000
 SAMPLES FOR GLOBAL SEARCH KERNEL (PSAMPLE_M1):          1
 SAMPLES FOR LOCAL SEARCH KERNEL (PSAMPLE_M2):           -1
 SAMPLES FOR LOCAL UNIVARIATE KERNEL (PSAMPLE_M3):       1
 METROPOLIS HASTINGS POPULATION SAMPLING FOR NON-GIBBS
 SAMPLED OMEGAS:
 SAMPLE ACCEPTANCE RATE (OACCEPT):                       0.500000000000000
 SAMPLES FOR GLOBAL SEARCH KERNEL (OSAMPLE_M1):          -1
 SAMPLES FOR LOCAL SEARCH KERNEL (OSAMPLE_M2):           -1
 SAMPLES FOR LOCAL UNIVARIATE SEARCH KERNEL (OSAMPLE_M3):-1
 MASS/IMP./POST. MATRIX REFRESH SETTING (MASSREST):      -1
 MASS MATRIX ACCUMULATION ITERATIONS (MADAPT):          -1
 MASS MATRIX BLOCKING TYPE (NUTS_MASS):                 B
 MODEL PARAMETERS TRASNFORMED BY MASS MATRIX (NUTS_TRANSFORM=0)
 POWER TERM WEIGHTING FOR MASS MATRIX ACCUM. (KAPPA):   1.00000000000000
 NUTS SAMPLE ACCEPTANCE RATE (NUTS_DELTA):                   0.800000000000000
 NUTS GAMMA SETTING (NUTS_GAMMA):                            5.000000000000000E-02
 DEG. FR. FOR T DIST.  PRIOR FOR THETAS (TTDF):        0.00000000000000
 DEG. FR. FOR LKJ CORRELATION PRIOR FOR OMEGAS (OLKJDF): 0.00000000000000
 WEIGHT FACTOR FOR STD PRIOR FOR OMEGAS (OVARF): 1.00000000000000
 DEG. FR. FOR LKJ CORRELATION PRIOR FOR SIGMAS (SLKJDF): 0.00000000000000
 WEIGHT FACTOR FOR STD PRIOR FOR SIGMAS (SVARF): 1.00000000000000
 NUTS WARMUP METHOD (NUTS_TEST):       0
 NUTS MAXIMAL DEPTH SEARCH (NUTS_MAXDEPTH):       10
 NUTS STAGE I WARMUP ITERATIONS (NUTS_INIT):       7.500000000000000E-02
 NUTS STAGE II base WARMUP ITERATIONS (NUTS_BASE): 2.500000000000000E-02
 NUTS STAGE III FINAL ITERATIONS (NUTS_TERM): 5.000000000000000E-02
 INITIAL ITERATIONS FOR STEP NUTS SIZE ASSESSMENT (NUTS_STEPITER): 1
 INTERVAL ITERATIONS FOR STEP NUTS SIZE ASSESSMENT (NUTS_STEPINTER):0
 ETA PARAMETERIZATION (NUTS_EPARAM):0
 OMEGA PARAMETERIZATION (NUTS_OPARAM):1
 SIGMA PARAMETERIZATION (NUTS_SPARAM):1
 NUTS REGULARIZING METHOD (NUTS_REG): 0.00000000000000

 THE FOLLOWING LABELS ARE EQUIVALENT
 PRED=PREDI
 RES=RESI
 WRES=WRESI
 IWRS=IWRESI
 IPRD=IPREDI
 IRS=IRESI

 EM/BAYES SETUP:
 THETAS THAT ARE MU MODELED:

 THETAS THAT ARE GIBBS SAMPLED:
   1   3   5   6   7   8   9  10  11  12  13  20  21
 THETAS THAT ARE METROPOLIS-HASTINGS SAMPLED:

 SIGMAS THAT ARE GIBBS SAMPLED:

 SIGMAS THAT ARE METROPOLIS-HASTINGS SAMPLED:

 OMEGAS ARE GIBBS SAMPLED

 MONITORING OF SEARCH:

 Burn-in Mode
 iteration         -180 MCMCOBJ=    10638.6549471017
billdenney commented 6 years ago

Perhaps more importantly, the execute suggests that it did not complete (the .lst file isn't copied into the directory with the .mod file).

billdenney commented 5 years ago

I just encountered this issue again with the following estimation lines:

$EST METHOD=ITS NITER=50 file=run13_its.ext
$EST METHOD=NUTS NBURN=200 NITER=500 PRINT=1 OLKJDF=3.0 file=run13.ext

Can you provide any guidance on how to help track down the problem or to work around the issue?

The new model also has the following apparent issue that I believe is NONMEM-related:

/tmp/modelfit_dir1/NM_run1>if [ -e psn.xml ];then cat psn.xml;else echo \<nofile/\>; fi

duplicate attribute at line 10993, column 73, byte 620886:
 nm:oaccept='0.500000000000000' nm:osample_m1='-1' nm:osample_m2='-1' nm:osample_m3='-1' nm:massreset='-1'
 nm:madapt='-1' nm:nuts_mass='b' nm:nuts_transform='0' nm:kappa='1.00000000000000'
 nm:nuts_delta='0.800000000000000' nm:nuts_gamma='5.000000000000000E-02' nm:nuts_transform='0'
========================================================================^
 nm:ttdf='0.00000000000000' nm:olkjdf='3.00000000000000' nm:ovarf='1.00000000000000'
 nm:slkjdf='0.00000000000000' nm:svarf='1.00000000000000' nm:nuts_test='0' nm:nuts_maxdepth='10'
 at /usr/lib/x86_64-linux-gnu/perl5/5.22/XML/Parser.pm line 187.
 Elapsed postprocess time in seconds:     0.24
 Elapsed finaloutput time in seconds:     0.02
 #CPUT: Total CPU Time in Seconds,    30050.907
NONMEM run failed. Check the lst-file in NM_run1 for errors
Not restarting this model.
F:1 ..
execute done
rikardn commented 5 years ago

I can reproduce your first problem now and it might be an issue with the PsN lst-file parser.

The workaround for a regular execute is to copy the lst-file and/or other resultfiles yourself from the NM_run1 directory and don't care about the PsN error message if you are confident that the estimation was fine.

I haven't seen your "duplicate attribute" error before. It seem to come from when NONMEM tries to create its xml results file. You could try the option -xmloff to NM-TRAN and see if that fixes the error.

rikardn commented 5 years ago

It was easy to fix. You can patch it yourself by checking the commit above.

billdenney commented 5 years ago

Thanks!

billdenney commented 5 years ago

I confirmed with Bob that the XML issue is a NONMEM issue. It may be fixed in 7.4.3 (I will update my NONMEM version to check after the current set of projects quiets down).