Jason-Conner-NOAA / MoBaSim

Code sharing repository for a project comparing model-based simulations of fisheries data to current design-based estimates.
1 stars 0 forks source link

getSurveyIdx throws error #1

Closed Jason-Conner-NOAA closed 3 years ago

Jason-Conner-NOAA commented 3 years ago

https://github.com/Jason-Conner-NOAA/MoBaSim/blob/7223ee8aa7e1d6c07669b7d22d6ae8f2add72fc4/GAM/gams.R#L129

Hi @casperwberg ,

I am running your gam.R code without BLAS (R version 4.02, SurveyIndex 1.09, DATRAS 1.01), and when fitting the model, I get this error:

6: stop("Error occured for age ", a, ".\n", "Try reducing the number of age groups or decrease the basis dimension of the smooths, k\n") 5: FUN(X[[i]], ...) 4: lapply(X, FUN, ...) 3: parallel::mclapply(1:noAges, do.one.a, mc.cores = mc.cores) 2: getSurveyIdx(ds[[SPECIES]], 1, myids = NULL, predD = allpd, cutOff = 0, fam = "Tweedie", modelP = fm, gamma = 1, control = list(trace = TRUE, maxit = 20)) 1: system.time(models[[SPECIES]] <- getSurveyIdx(ds[[SPECIES]], 1, myids = NULL, predD = allpd, cutOff = 0, fam = "Tweedie", modelP = fm, gamma = 1, control = list(trace = TRUE, maxit = 20)))

Have you run into this and do you have suggestions? I installed RhpcBLASctl, but I don't know how to install openBLAS, and I believe the code should work in serial.

Thank you

casperwberg commented 3 years ago

HI Jason,

Very recently we fixed an issue in the DATRAS package (it was not working properly with newer R versions). Could you please try to re-install DATRAS and see if this fixes the problem?

Wrt. openBLAS, actually an even better (and easier) option may be to install Microsoft R open (this includes optimized BLAS libraries as well). This works on both Windows and Linux. I would highly recommend this both for the GAMs and VAST as it gives quite drastic speed-ups.

Kind regards, Casper

Den lør. 14. nov. 2020 kl. 00.26 skrev Jason Conner < notifications@github.com>:

https://github.com/Jason-Conner-NOAA/MoBaSim/blob/7223ee8aa7e1d6c07669b7d22d6ae8f2add72fc4/GAM/gams.R#L129

Hi @casperwberg https://github.com/casperwberg ,

I am running your gam.R code without BLAS (R version 4.02, SurveyIndex 1.09, DATRAS 1.01), and when fitting the model, I get this error:

6: stop("Error occured for age ", a, ".\n", "Try reducing the number of age groups or decrease the basis dimension of the smooths, k\n") 5: FUN(X[[i]], ...) 4: lapply(X, FUN, ...) 3: parallel::mclapply(1:noAges, do.one.a, mc.cores = mc.cores) 2: getSurveyIdx(ds[[SPECIES]], 1, myids = NULL, predD = allpd, cutOff = 0, fam = "Tweedie", modelP = fm, gamma = 1, control = list(trace = TRUE, maxit = 20)) 1: system.time(models[[SPECIES]] <- getSurveyIdx(ds[[SPECIES]], 1, myids = NULL, predD = allpd, cutOff = 0, fam = "Tweedie", modelP = fm, gamma = 1, control = list(trace = TRUE, maxit = 20)))

Have you run into this and do you have suggestions? I installed RhpcBLASctl, but I don't know how to install openBLAS, and I believe the code should work in serial.

Thank you

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Jason-Conner-NOAA/MoBaSim/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRWYZCJHGPR327LGX4XP3SPW6AXANCNFSM4TVE5V6Q .

Jason-Conner-NOAA commented 3 years ago

Hi Casper,

Thanks for the suggestions.

Reinstalling DATRAS (either from github or rforge.net) did not solve the issue. Nor did running the code using MRAN 3.5.1.

Best wishes, Jason

casperwberg commented 3 years ago

Ok, use the github version - the rforge.net version is old and no longer maintained.

Were there any changes to the data file since I ran it? I see the data file is not on the github, so I'm not able to test it myself. Could we add the data file to the github or is it too large?

Cheers, Casper

Den tir. 17. nov. 2020 kl. 22.25 skrev Jason Conner < notifications@github.com>:

Hi Casper,

Thanks for the suggestions.

Reinstalling DATRAS (either from github or rforge.net) did not solve the issue. Nor did running the code using MRAN 3.5.1.

Best wishes, Jason

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Jason-Conner-NOAA/MoBaSim/issues/1#issuecomment-729223172, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRWY447SPPLOG3VRMYIDDSQLS3RANCNFSM4TVE5V6Q .

Jason-Conner-NOAA commented 3 years ago

Apologies, Casper. Yes I have updated the data file. data and shapefiles data are available from the Google drive. Please use /R/pull_data_googledrive.R to access the updated data files, then use: load(here::here("data","EBSbundle_1_2.rdata")) to load the data. I will be making a smaller dataset of 5 years and including that on the Google drive as well.

jc

casperwberg commented 3 years ago

Ok, no problem, thanks. The errors were due to renaming of lat/long variables (e.g. START_LATITUDE to LAT_DEGREES) and because there are some NA's in GEAR_TEMPERATURE in the updated data set. I have updated the script so it works now, and I've also updated the surveyIndex package with a new argument "CIlevel" discussed in our last meeting, so please re-install before testing.

Cheers, Casper

Den tor. 19. nov. 2020 kl. 02.45 skrev Jason Conner < notifications@github.com>:

Apologies, Casper. Yes I have updated the data file. data and shapefiles data are available from the Google drive. Please use /R/pull_data_googledrive.R http://R/pull_data_googledrive.R to access the updated data files, then use: load(here::here("data","EBSbundle_1_2.rdata")) to load the data. I will be making a smaller dataset of 5 years and including that on the Google drive as well.

jc

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Jason-Conner-NOAA/MoBaSim/issues/1#issuecomment-730070772, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRWY5Y7EZIRQNZ776O773SQR2CVANCNFSM4TVE5V6Q .

Jason-Conner-NOAA commented 3 years ago

That worked! Thank you Casper.