search

bachmannpatrick / CLVTools

R-Package for estimating CLV
54 stars 14 forks source link

Model Convergence #149

Closed rlumor closed 3 years ago

rlumor commented 4 years ago

Under what definitions will the pnbd model not converge?

setDTthreads(1) registerDoFuture() plan("multisession") est.pnbd <- pnbd(clv.data = clv_train,optimx.args = list(method="Nelder-Mead"))

Results

setDTthreads(1) registerDoFuture() plan("multisession") est.pnbd <- pnbd(clv.data = clv_train,optimx.args = list(method="Nelder-Mead")) Starting estimation... Estimation finished! Warning message: In optimx.run(par, optcfg$ufn, optcfg$ugr, optcfg$uhess, lower, : Gradient not computable after method Nelder-Mead

pschil commented 4 years ago

Under what definitions will the pnbd model not converge?

Im not aware of any specific conditions under which the pnbd model does or does not converge. From your output it looks as if the optimization did in fact converge (or maybe reached the maximum number of iterations, check the "convcode" in est.pnbd@optimx.estimation.output and compare it to ?optimx). The message "Gradient not computable after method Nelder-Mead" means that the gradient at the final parameter set could (partially) not be derived because the LogLikelihood function returned a non-finite value (= NaN or Inf) when numerically calculating the gradient. Therefore, the hessian (est.pnbd@optimx.hessian) will have non-finite values and the variance-covariance matrix which is based on it cannot be derived what means there are no standard errors, p-values, KKTs, etc. Nevertheless, you should be able to predict.

plan("multisession")

Are you fitting the pnbd model with dynamic covariates? Only the dynamic covariates model is parallelized, the standard model (incl the static covariates) does not benefit from parallelization through doParallel.

rlumor commented 4 years ago

Under what definitions will the pnbd model not converge?

Im not aware of any specific conditions under which the pnbd model does or does not converge. From your output it looks as if the optimization did in fact converge (or maybe reached the maximum number of iterations, check the "convcode" in est.pnbd@optimx.estimation.output and compare it to ?optimx). The message "Gradient not computable after method Nelder-Mead" means that the gradient at the final parameter set could (partially) not be derived because the LogLikelihood function returned a non-finite value (= NaN or Inf) when numerically calculating the gradient. Therefore, the hessian (est.pnbd@optimx.hessian) will have non-finite values and the variance-covariance matrix which is based on it cannot be derived what means there are no standard errors, p-values, KKTs, etc. Nevertheless, you should be able to predict.

plan("multisession")

Are you fitting the pnbd model with dynamic covariates? Only the dynamic covariates model is parallelized, the standard model (incl the static covariates) does not benefit from parallelization through doParallel.

the pnbd model generated NaN values for the DERT, there NaN CLTV for some of the customers so I assumed it was because of the model not converging

pschil commented 4 years ago

the pnbd model generated NaN values for the DERT, there NaN CLTV for some of the customers so I assumed it was because of the model not converging

This is happens mostly because the estimated parameters (likely beta) are very large which then leads to issues with numerical stability when calculating the hypergeometric 1F1 function in DERT.