Erroneous results using jagsUI but not R2jags

[jhncsu]

Our research group has been using R code to generate replicate simulated data sets and estimating parameters using JAGS. The purpose is to evaluate the reliability of parameter estimates when varying the experimental design (e.g., sample size). The estimates vary in a normal fashion among replicates when using R2jags, but runs using jagsUI are initially normal then degenerate into a repetitive pattern. This holds whether done in RStudio or directly in R, and whether using a standard jags call or autojags, and whether using parallel processing or not. It occurs on multiple PCs. We are using the latest versions of R, JAGS, rjags, jagsUI, and R2jags.

The biological model is referred to as an N-mixture model, used to estimate abundance based on repeated surveys of individual sites. But we have obtained the same sort of aberrant results with a completely different biological model. The common thread is the use of a large number of replicate calls to JAGS, as we do not get aberrant results with fewer replicates (fewer than 200 or 500).

An example of the code is attached. This plot illustrates the pattern from a run using jagsUI.

We would welcome any insights!

N-mixtureExample_jagsUI.txt

[kenkellner]

I was able to replicate your issue. From what I can tell now something strange is happening with the random number generator in R such that I am getting a cyclical pattern in random seeds which results in the cyclical pattern of parameter estimates (and presumably also values of N and Count) seen in your figure.

Thus I suspect that jagsUI is not producing "incorrect" results per se but rather is being fed (over the course of the simulation iterations) a cycle of the same data and same random seed to start the MCMC chains, resulting in the same estimated parameter values.

I was able to get results that seem correct to me by manually specifying a set of random seeds before starting the simulation loop, as shown in the attached modified version of your code. While this clearly does not solve the issue it may be a temporary workaround, and I'd be interested to know if it also works for you.

N-mixtureExample_jagsUI_kfk.txt

[jhncsu]

Yes, that seems to fix it. Thank you for the quick workaround!

Any idea why the random seed issue occurs with jagsUI but not R2jags?

[kenkellner]

For reasons I still don't fully understand, the random number being fed into the default seed argument was limited to a small, cyclical set of possible values instead of being completely random. This cyclical setting of the random seed was then being carried over to generation of the realizations of N and Count in the simulation, resulting in only ~5-10 unique realizations of the data (instead of 500). Since R2jags doesn't manually set the random seed, it wasn't running into this issue.

By manually specifying the seeds as I did in the code I attached above, you break this cyclical pattern and get the correct results. Going forward, I am addressing this problem by changing the default way of generating a seed from runif(1,1,10000) to as.integer(Sys.time()). Based on my tests this also avoids any cyclical random number issues, since the generation of the new random seed for the new analysis is no longer dependent on the seed set on the previous iteration of the simulation (instead depends on your clock).

You can see what I changed in 0341735. I've also attached a new draft version of the package - I'd appreciate if you could try it and let me know if it fixes your issue. If you try it, make sure in your code to remove the old "seed" argument when you call jags() to allow the function to use the new default.

jagsUI_1.4.4.tar.gz

[jhncsu]

I did a 500-replicate test using the new draft version of jagsUI and everything looks nicely random: Thanks very much for resolving this issue!

[kenkellner]

Excellent, thanks so much for reporting this issue!