stan() (occasionally) returns nothing with no errors

[hhau]

Summary:

Running code such as model_fit <- stan() occasionally results in no assignment / output, despite Stan appearing to run normally. In fact the variable model_fit does not exist in the environment.

Description:

I call rstan::stan() with multiple chains from within Rstudio, and assign it to some variable. Upon the completion of the sampling process, either the assignment doesn't occur, or something goes awry in the return process.

This might have something to do with:

• The long runtime of my model (and the fact I run said long model from within an Rstudio session, as opposed to calling Rscript on the fitter.R file)
• The enterprise environment in which I run Stan

I realise that this issue is going to be practically impossible to track down (and might even be specific to my environment), but I would feel remiss not to mention it here. (I'm also somewhat irked as I just lost a model that ran for 90,000 seconds :/ ).

Reproducible Steps:

I have attached the stan model with which I encountered the issue, as well as a script to generate data. (Unable to share data in R code detailed below), and the script used to fit the model.

2018-04-18_double-tanh-log-time-series.zip

I'm going to keep trying to reproduce this, and will update this thread if I can find something that breaks consistently.

Current Output:

If applicable, any relevant output from RStan.

Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> setwd(".../projects/2018-04-18_double-tanh-log-time-series/")
> library(rstan)
Loading required package: ggplot2
Loading required package: StanHeaders
rstan (Version 2.17.3, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)
> library(ddcurves2)
Loading required package: Rcpp
> stan_dat <- list(n_depths = length(Crux_KP150_Phs1$depths),
+                  n_times = nrow(Crux_KP150_Phs1$density_mat),
+                  depths = Crux_KP150_Phs1$depths,
+                  densities = Crux_KP150_Phs1$density_mat)
> prefit <- stan(file = "../../projects/ddcurves2/src/stan_files/double_tanh_time_series.stan",
+                chains = 0, 
+                data = stan_dat)
DIAGNOSTIC(S) FROM PARSER:
WARNING: left-hand side variable (name=beta_orig) occurs on right-hand side of assignment, causing inefficient deep copy to avoid aliasing.

In file included from C:/Program Files/R/R-3.4.3/library/BH/include/boost/config.hpp:39:0,
                 from C:/Program Files/R/R-3.4.3/library/BH/include/boost/math/tools/config.hpp:13,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math/rev/core/var.hpp:7,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math/rev/core.hpp:12,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math/rev/mat.hpp:4,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math.hpp:4,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/src/stan/model/model_header.hpp:4,
                 from filea2fc75ef7c06.cpp:8:
C:/Program Files/R/R-3.4.3/library/BH/include/boost/config/compiler/gcc.hpp:186:0: warning: "BOOST_NO_CXX11_RVALUE_REFERENCES" redefined
 #  define BOOST_NO_CXX11_RVALUE_REFERENCES
 ^
<command-line>:0:0: note: this is the location of the previous definition
In file included from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math/rev/core.hpp:44:0,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math/rev/mat.hpp:4,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math.hpp:4,
                 from C:/Program Files/R/R-3.4.3/library/StanHeaders/include/src/stan/model/model_header.hpp:4,
                 from filea2fc75ef7c06.cpp:8:
C:/Program Files/R/R-3.4.3/library/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints.hpp:14:17: warning: 'void stan::math::set_zero_all_adjoints()' defined but not used [-Wunused-function]
     static void set_zero_all_adjoints() {
                 ^
the number of chains is less than 1; sampling not done
> phi_prime <- rbeta(6, 6, 1)
> kappa_prime <- (1 - phi_prime) * log(c(1023, 1.1, 50, 42, 100, 45))
> init_sub <- list(beta_prime_t0 = log(rlnorm(6, meanlog = log(c(1023, 1.1, 50, 42, 100, 45)), sdlog = 0.001)),
+                  phi_prime = phi_prime,
+                  kappa_prime = kappa_prime,
+                  beta_prime_sigma = rlnorm(6, meanlog = log(c(0.18, 0.12, 26.1, 8.09, 12.6, 6.65)), sdlog = 0.002),
+                  innovations_prime = array(0, dim = c(737 - 1, 6)), 
+                  sigma_curve = rlnorm(1, meanlog = log(0.1), sdlog = 0.2))
> model_fit <- stan(fit = prefit,
+                   data = stan_dat,
+                   iter = 120,
+                   warmup = 100,
+                   save_warmup = FALSE,
+                   chains = 3,
+                   cores = 3,
+                   control = list(adapt_delta = 0.95,
+                                  max_treedepth = 20),
+                   verbose = TRUE,
+                   refresh = 1
+                   )

CHECKING DATA AND PREPROCESSING FOR MODEL 'double_tanh_time_series' NOW.

COMPILING MODEL 'double_tanh_time_series' NOW.

STARTING SAMPLER FOR MODEL 'double_tanh_time_series' NOW.
Loading required namespace: rstudioapi

> saveRDS(model_fit,
+         file = paste0("../../results/",
+                       format(Sys.time(), "%F_%H-%M-%S"),
+                       "_double-tanh-time-series-log-test.rds"))
Error in saveRDS(model_fit, file = paste0("../../results/", format(Sys.time(),  : 
  object 'model_fit' not found

I have attached the stanprogress file from the tempdir(), which indicates that stan seemed to run and complete normally. filea2fc2f677584_StanProgress.txt

Expected Output:

model_fit to exist as a stanfit object, with samples. Or an error that indicates that the return process / assignment failed.

RStan Version:

[1] ‘2.17.3’

R Version:

[1] "R version 3.4.3 (2017-11-30)"

Operating System:

Windows 7 Enterprise, SP1.

[jgabry]

Thanks for reporting this and sorry for the hassle! You’re right that this seems hard to pin down. I don’t think this is the first time this has been reported but not something I’ve ever been able to reproduce. Please do keep us posted if you figure anything out. You might also try asking on the Stan Forums (discourse.mc-stan.org) in case one of our users has experienced this and has any insights.

[aaronjg]

I have had issues where a Stan run that is aborted with C-c has redirected the R output to some file in the temporary directory, and then ALL errors from that R session are lost to the ether (well, actually they are printed into that file, and if you track that down with lsof, you can find them again). Could this be related?

[ifellows]

I ran into this issue as well. I determined that in my case the problem likely stemmed from an out of memory issue. I had plenty of RAM (64gb), but i suspect somewhere a vector/array was being allocated with too many elements. In some cases when I encountered the problem nothing would be returned, but when I quit R (in the terminal) I would get Segmentation fault: 11.

In particular I had 2 transformed parameters and ~6 generated quantities of length 53431. When I paired down the generated quantities to 1, the error went away.