James-Thorson-NOAA / VAST

Spatio-temporal analysis of univariate or multivariate data, e.g., standardizing data for multiple species or stages
http://www.FishStats.org
GNU General Public License v3.0
124 stars 54 forks source link

Index calculations with random walks on #17

Closed GreatDana closed 7 years ago

GreatDana commented 7 years ago

Fitting the Gulf of Alaska trawl survey data (with two species) and wanting to fill in the years between triennial and biennial surveys. I am using a RhoConfig of c(2,2,2,2). Looking at the raw data files it is doing something, but not really sensible between years. The SpatialDeltaGLMM::PlotIndex_Fn doesn't seem to work when use_bias_corr is on. The plot is of the two indices without bias correction. The plotIndex function looks for a column called unbiased in Sdreport, which appears to have this column but only NAs. The optimization was done as so:

Opt = TMBhelper::Optimize( obj=Obj, lower=TmbList[["Lower"]], upper=TmbList[["Upper"]], getsd=TRUE, savedir=DateFile, bias.correct=TRUE,newtonsteps=0)

In the TMBHelper::Optimize function I can't figure out any way that the option "bias.correct" =TRUE is utilized but that might be my failure to understand the code. Hopefully others have had better luck with data that is not annual. I probably should only be doing a single species example here, but it should be the same issue.

Thanks all, Cheers, Dana

index

James-Thorson commented 7 years ago

@GreatDana,

Thanks for exploring VAST. I agree that using a single model to estimate abundance and smooth across years when applying an Fmsy/M estimator of OFL makes more sense than fitting a model and then applying a smoother post-hoc. And it certainly seems interesting to do this using data from multiple species simultaneously, although Thorson and Barnett 2017 suggests that you might not seem much shrinkage in predictive interval estimates.

In answer to your question, bias.correct=TRUE when passed to TMBhelper::Optimize is passed through to TMB::sdreport via the ... input.

In response to your larger comment, all versions of the VAST CPP had a small bug in line log_mean_D_ctl = log( log_mean_D_ctl );, which is now fixed to log_mean_D_ctl = log( mean_D_ctl );. This bug meant that log_mean_D_ctl was equal to -Inf, which caused a numerical problem during bias-correction, such that no variable was being bias-corrected whenever Options["Calculate_effective_area"]=1. I have now fixed this, and also added a logical check to SpatialDeltaGLMM::PlotIndex_Fn where it now checks for bias-correction, and if all bias-corrected values are NA then it errors out with a message to report the problem to me.

Anyway, applied to the GOA Pacific cod data set that comes with SpatialDeltaGLMM, using RhoConfig = c("Beta1"=2, "Beta2"=2, "Epsilon1"=2, "Epsilon2"=2), and turning on bias.correct=TRUE using n_x=50, I get the index: index

and COG center_of_gravity

where both are clearly smoothing across years with no sampling data, but increasing the standard error for those years. As always, to get the updates you will have to delete VAST and SpatialDeltaGLMM and re-install VAST (which auto-reinstalls the dependency SpatialDeltaGLMM)

Jim

GreatDana commented 7 years ago

Hi again, issue unfortunately not closed for me: I am getting a different error now:

Error in sparseHessianFun(env, skipFixedEffects = skipFixedEffects) : Memory allocation fail in function 'MakeADHessObject2'


Error: attempt to use zero-length variable name

When calling it like this:

Opt = TMBhelper::Optimize( obj=Obj, lower=TmbList[["Lower"]], upper=TmbList[["Upper"]], getsd=TRUE, savedir=DateFile, bias.correct=TRUE,newtonsteps=0)

This model runs when bias.correct=FALSE, or getsd=FALSE, or both = FALSE.... But if getsd=FALSE it doesn't run the index plot anyway.

I reinstalled VAST and after removing spatialDeltaGLMM and also reinstalled TMB and the example works fine.

Cheers, Dana

James-Thorson commented 7 years ago

It's just saying you have too little memory on your machine given the number of knots. Please try again with many fewer knots to check k if that solves it, or try again on machine with 32-64 GB of memory

On Feb 16, 2017 3:30 PM, "Dana Hanselman" notifications@github.com wrote:

Hi again, issue unfortunately not closed for me: I am getting a different error now:

Error in sparseHessianFun(env, skipFixedEffects = skipFixedEffects) : Memory allocation fail in function 'MakeADHessObject2'

Error: attempt to use zero-length variable name

When calling it like this:

Opt = TMBhelper::Optimize( obj=Obj, lower=TmbList[["Lower"]], upper=TmbList[["Upper"]], getsd=TRUE, savedir=DateFile, bias.correct=TRUE,newtonsteps=0)

This model runs when bias.correct=FALSE, or getsd=FALSE, or both = FALSE.... But if getsd=FALSE it doesn't run the index plot anyway.

I reinstalled VAST and after removing spatialDeltaGLMM and also reinstalled TMB and the example works fine.

Cheers, Dana

— You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub https://github.com/James-Thorson/VAST/issues/17#issuecomment-280498727, or mute the thread https://github.com/notifications/unsubscribe-auth/AHnqTUKONy5vKoXrw5T61U2GkN9qYSLHks5rdNwogaJpZM4L-Cya .