search

fishfollower / SAM

Git page for SAM R-package
47 stars 23 forks source link

sam.fit issue #77

Closed h-du-pontavice closed 1 month ago

h-du-pontavice commented 1 month ago

Hello,

I'm having a problem running the function sam.fit with my mac (MacOS Sonoma 14.6.1).

I ran sam.fit using my own data, which worked on my old Windows laptop, but the R session crashes when I run sam.fit on my Mac. The R session also crashes when I use the example presented on the README of the stockassessment R package.

After digging into this issue, I found that the problem probably comes from the TMB function MakeADFun. So, I checked the TMB installation as presented here but everything works well.

Here is the info on the session : 

Platform: aarch64-apple-darwin20
Running under: macOS Sonoma 14.6.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/Paris
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] stockassessment_0.12.0

loaded via a namespace (and not attached):
[1] compiler_4.4.1    Matrix_1.7-0      parallel_4.4.1    tools_4.4.1       TMB_1.9.15       
[6] rstudioapi_0.16.0 ellipse_0.5.0     grid_4.4.1        lattice_0.22-6
h-du-pontavice commented 1 month ago

After digging more into this issue, the problem likely comes from the function :

getParameterOrder <- function(data, parameters, reportenv, DLL) {
  control <- NULL
  .Call("getParameterOrder", data, parameters, control, PACKAGE=DLL)
}

Do you any idea to solve this issue ?

fishfollower commented 1 month ago

Thank you for reporting this and looking into it.

I cannot reproduce it because I don't have access to a mac.

I have been told that macs can treat NA/NaN a bit differently, so possibly it is connected to that? Does it also crash with:

example(sam.fit)

Is it possible for you to run a backtrace and send the results back?

R -d lldb run source("script.R") bt

Thanks,

Anders.

h-du-pontavice commented 1 month ago

Thanks for your reply. Yes, it also crashes with example(sam.fit).

I run a backtrace for example(sam.fit) with : R -d lldb run source("script.R") bt

Here is the result : 

> source("test/sam_test2.R")

sam.ft> data(nscodData)

sam.ft> data(nscodConf)

sam.ft> data(nscodParameters)

sam.ft> fit <- sam.fit(nscodData, nscodConf, nscodParameters, silent = TRUE)
TMB has received an error from Eigen. The following condition was not met:
index >= 0 && index < size()
Please check your matrix-vector bounds etc., or run your program through a debugger.
Process 8465 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x00000001854b6600 libsystem_kernel.dylib`__pthread_kill + 8
libsystem_kernel.dylib`__pthread_kill:
->  0x1854b6600 <+8>:  b.lo   0x1854b6620    ; <+40>
    0x1854b6604 <+12>: pacibsp 
    0x1854b6608 <+16>: stp    x29, x30, [sp, #-0x10]!
    0x1854b660c <+20>: mov    x29, sp
Target 0: (R) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00000001854b6600 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x00000001854eef70 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x00000001853fb908 libsystem_c.dylib`abort + 128
    frame #3: 0x000000010a929e00 stockassessment.so`double nllObs<double>(dataSet<double>&, confSet&, paraSet<double>&, forecastSet<double>&, tmbutils::array<double>&, tmbutils::array<double>&, tmbutils::array<double>&, Recruitment<double>&, MortalitySet<double>&, data_indicator<tmbutils::vector<double>, double>&, int, objective_function<double>*) + 19584
    frame #4: 0x000000010abbb370 stockassessment.so`objective_function<double>::operator()() + 13552
    frame #5: 0x000000010ab75a1c stockassessment.so`getParameterOrder + 92
    frame #6: 0x0000000100593a20 libR.dylib`R_doDotCall(fun=<unavailable>, nargs=4, cargs=0x000000016fdf5838, call=0x000000010e9fb4d8) at dotcode.c:763:11 [opt]
    frame #7: 0x00000001005948fc libR.dylib`do_dotcall(call=0x000000010e9fb4d8, op=<unavailable>, args=<unavailable>, env=<unavailable>) at dotcode.c:1437:11 [opt]
    frame #8: 0x00000001005dab4c libR.dylib`bcEval_loop(ploc=0x000000016fdf67f8) at eval.c:8141:14 [opt]
    frame #9: 0x00000001005c346c libR.dylib`bcEval(body=0x000000010e709020, rho=0x000000010e7dac30) at eval.c:7524:16 [opt]
    frame #10: 0x00000001005c2b6c libR.dylib`Rf_eval(e=0x000000010e709020, rho=0x000000010e7dac30) at eval.c:1167:8 [opt]
    frame #11: 0x00000001005c572c libR.dylib`R_execClosure(call=0x000000010e7db410, newrho=0x000000010e7dac30, sysparent=<unavailable>, rho=0x0000000104f022d0, arglist=<unavailable>, op=0x000000010e70a6e0) at eval.c:2398:22 [opt]
    frame #12: 0x00000001005c4824 libR.dylib`applyClosure_core(call=0x000000010e7db410, op=0x000000010e70a6e0, arglist=0x000000010e7db368, rho=0x0000000104f022d0, suppliedvars=<unavailable>, unpromise=TRUE) at eval.c:2311:16 [opt]
    frame #13: 0x00000001005c2e08 libR.dylib`Rf_eval [inlined] Rf_applyClosure(call=0x000000010e7db410, op=0x000000010e70a6e0, arglist=<unavailable>, rho=0x0000000104f022d0, suppliedvars=<unavailable>, unpromise=TRUE) at eval.c:2333:16 [opt]
    frame #14: 0x00000001005c2df4 libR.dylib`Rf_eval(e=0x000000010e7db410, rho=0x0000000104f022d0) at eval.c:1285:12 [opt]
    frame #15: 0x0000000100561484 libR.dylib`do_docall(call=0x000000010e7db410, op=<unavailable>, args=0x000000010e68e568, rho=<unavailable>) at coerce.c:2761:12 [opt]
    frame #16: 0x00000001005dab4c libR.dylib`bcEval_loop(ploc=0x000000016fdf7e08) at eval.c:8141:14 [opt]
    frame #17: 0x00000001005c346c libR.dylib`bcEval(body=0x0000000104edef58, rho=0x0000000104f022d0) at eval.c:7524:16 [opt]
    frame #18: 0x00000001005c2b6c libR.dylib`Rf_eval(e=0x0000000104edef58, rho=0x0000000104f022d0) at eval.c:1167:8 [opt]
    frame #19: 0x00000001005c572c libR.dylib`R_execClosure(call=0x0000000104df8208, newrho=0x0000000104f022d0, sysparent=<unavailable>, rho=0x000000012b84d0b8, arglist=<unavailable>, op=0x0000000104edfc78) at eval.c:2398:22 [opt]
    frame #20: 0x00000001005c4824 libR.dylib`applyClosure_core(call=0x0000000104df8208, op=0x0000000104edfc78, arglist=0x0000000104f027a0, rho=0x000000012b84d0b8, suppliedvars=<unavailable>, unpromise=TRUE) at eval.c:2311:16 [opt]
    frame #21: 0x00000001005c2e08 libR.dylib`Rf_eval [inlined] Rf_applyClosure(call=0x0000000104df8208, op=0x0000000104edfc78, arglist=<unavailable>, rho=0x000000012b84d0b8, suppliedvars=<unavailable>, unpromise=TRUE) at eval.c:2333:16 [opt]
    frame #22: 0x00000001005c2df4 libR.dylib`Rf_eval(e=0x0000000104df8208, rho=0x000000012b84d0b8) at eval.c:1285:12 [opt]
    frame #23: 0x00000001005c8fe8 libR.dylib`do_set(call=<unavailable>, op=0x000000012b812808, args=0x0000000104df8198, rho=0x000000012b84d0b8) at eval.c:3582:8 [opt]
    frame #24: 0x00000001005c2d34 libR.dylib`Rf_eval(e=0x0000000104df8160, rho=0x000000012b84d0b8) at eval.c:1237:12 [opt]
    frame #25: 0x00000001005ca29c libR.dylib`do_eval(call=<unavailable>, op=<unavailable>, args=<unavailable>, rho=<unavailable>) at eval.c:3974:9 [opt]
    frame #26: 0x00000001005dab4c libR.dylib`bcEval_loop(ploc=0x000000016fdf9a88) at eval.c:8141:14 [opt]
    frame #27: 0x00000001005c346c libR.dylib`bcEval(body=0x000000010b82e500, rho=0x0000000104deae48) at eval.c:7524:16 [opt]
    frame #28: 0x00000001005c2b6c libR.dylib`Rf_eval(e=0x000000010b82e500, rho=0x0000000104deae48) at eval.c:1167:8 [opt]
    frame #29: 0x00000001005c36a8 libR.dylib`forcePromise(e=0x0000000104ee01f0) at eval.c:976:13 [opt]
    frame #30: 0x00000001005c2bd4 libR.dylib`Rf_eval(e=0x000000012b866120, rho=0x0000000104ee0148) at eval.c:1192:6 [opt]
    frame #31: 0x00000001005ca380 libR.dylib`do_withVisible(call=<unavailable>, op=<unavailable>, args=0x000000010301f848, rho=0x0000000104ee0148) at eval.c:3999:9 [opt]
    frame #32: 0x0000000100633190 libR.dylib`do_internal(call=<unavailable>, op=<unavailable>, args=<unavailable>, env=0x0000000104ee0148) at names.c:1409:11 [opt]
    frame #33: 0x00000001005dad7c libR.dylib`bcEval_loop(ploc=0x000000016fdfade8) at eval.c:8161:15 [opt]
    frame #34: 0x00000001005c346c libR.dylib`bcEval(body=0x0000000109f70e70, rho=0x0000000109f915f8) at eval.c:7524:16 [opt]
    frame #35: 0x00000001005c2b6c libR.dylib`Rf_eval(e=0x0000000109f70e70, rho=0x0000000109f915f8) at eval.c:1167:8 [opt]
    frame #36: 0x00000001005c572c libR.dylib`R_execClosure(call=0x0000000103020370, newrho=0x0000000109f915f8, sysparent=<unavailable>, rho=0x000000012b84d0b8, arglist=<unavailable>, op=0x0000000109f71b20) at eval.c:2398:22 [opt]
    frame #37: 0x00000001005c4824 libR.dylib`applyClosure_core(call=0x0000000103020370, op=0x0000000109f71b20, arglist=0x0000000109f91978, rho=0x000000012b84d0b8, suppliedvars=<unavailable>, unpromise=TRUE) at eval.c:2311:16 [opt]
    frame #38: 0x00000001005c2e08 libR.dylib`Rf_eval [inlined] Rf_applyClosure(call=0x0000000103020370, op=0x0000000109f71b20, arglist=<unavailable>, rho=0x000000012b84d0b8, suppliedvars=<unavailable>, unpromise=TRUE) at eval.c:2333:16 [opt]
    frame #39: 0x00000001005c2df4 libR.dylib`Rf_eval(e=0x0000000103020370, rho=0x000000012b84d0b8) at eval.c:1285:12 [opt]
    frame #40: 0x00000001005ca29c libR.dylib`do_eval(call=<unavailable>, op=<unavailable>, args=<unavailable>, rho=<unavailable>) at eval.c:3974:9 [opt]
    frame #41: 0x00000001005dab4c libR.dylib`bcEval_loop(ploc=0x000000016fdfc5d8) at eval.c:8141:14 [opt]
    frame #42: 0x00000001005c346c libR.dylib`bcEval(body=0x000000010b82e500, rho=0x000000010b82d9d8) at eval.c:7524:16 [opt]
    frame #43: 0x00000001005c2b6c libR.dylib`Rf_eval(e=0x000000010b82e500, rho=0x000000010b82d9d8) at eval.c:1167:8 [opt]
    frame #44: 0x00000001005c36a8 libR.dylib`forcePromise(e=0x0000000109f72060) at eval.c:976:13 [opt]
    frame #45: 0x00000001005c2bd4 libR.dylib`Rf_eval(e=0x000000012b866120, rho=0x0000000109f71fb8) at eval.c:1192:6 [opt]
    frame #46: 0x00000001005ca380 libR.dylib`do_withVisible(call=<unavailable>, op=<unavailable>, args=0x000000010301f848, rho=0x0000000109f71fb8) at eval.c:3999:9 [opt]
    frame #47: 0x0000000100633190 libR.dylib`do_internal(call=<unavailable>, op=<unavailable>, args=<unavailable>, env=0x0000000109f71fb8) at names.c:1409:11 [opt]
    frame #48: 0x00000001005dad7c libR.dylib`bcEval_loop(ploc=0x000000016fdfd938) at eval.c:8161:15 [opt]
    frame #49: 0x00000001005c346c libR.dylib`bcEval(body=0x000000010b80e658, rho=0x000000010b82d9d8) at eval.c:7524:16 [opt]
    frame #50: 0x00000001005c2b6c libR.dylib`Rf_eval(e=0x000000010b80e658, rho=0x000000010b82d9d8) at eval.c:1167:8 [opt]
    frame #51: 0x00000001005c572c libR.dylib`R_execClosure(call=0x000000012bf125c0, newrho=0x000000010b82d9d8, sysparent=<unavailable>, rho=0x000000012b84d0b8, arglist=<unavailable>, op=0x000000012bf12358) at eval.c:2398:22 [opt]
    frame #52: 0x00000001005c4824 libR.dylib`applyClosure_core(call=0x000000012bf125c0, op=0x000000012bf12358, arglist=0x000000010b82ddc8, rho=0x000000012b84d0b8, suppliedvars=<unavailable>, unpromise=TRUE) at eval.c:2311:16 [opt]
    frame #53: 0x00000001005c2e08 libR.dylib`Rf_eval [inlined] Rf_applyClosure(call=0x000000012bf125c0, op=0x000000012bf12358, arglist=<unavailable>, rho=0x000000012b84d0b8, suppliedvars=<unavailable>, unpromise=TRUE) at eval.c:2333:16 [opt]
    frame #54: 0x00000001005c2df4 libR.dylib`Rf_eval(e=0x000000012bf125c0, rho=0x000000012b84d0b8) at eval.c:1285:12 [opt]
    frame #55: 0x000000010061f034 libR.dylib`Rf_ReplIteration(rho=0x000000012b84d0b8, savestack=<unavailable>, browselevel=<unavailable>, state=0x000000016fdfe1f0) at main.c:262:2 [opt]
    frame #56: 0x0000000100620668 libR.dylib`R_ReplConsole(rho=0x000000012b84d0b8, savestack=0, browselevel=0) at main.c:314:11 [opt]
    frame #57: 0x00000001006205a4 libR.dylib`run_Rmainloop at main.c:1216:5 [opt]
    frame #58: 0x0000000100620710 libR.dylib`Rf_mainloop at main.c:1223:5 [opt]
    frame #59: 0x0000000100003ea0 R`main + 32
    frame #60: 0x000000018516c274 dyld`start + 2840
calbertsen commented 1 month ago

What is the output if you run: R -d lldb run source("script.R") bt f 3 fr v

And does it still fail if you modify your R code to:

data(nscodData) data(nscodConf) nscodConf$fixVarToWeight <- c(0,0,0) data(nscodParameters) fit <- sam.fit(nscodData, nscodConf, nscodParameters, silent = TRUE)

h-du-pontavice commented 1 month ago

Thanks for your help with this issue.

Yes, it still fails with the modified code you sent.

Here is the output for the backtrace (with f 3 and fr v) with example(sam.fit) :

source("sam_test2.R")

sam.ft> data(nscodData)

sam.ft> data(nscodConf)

sam.ft> data(nscodParameters)

sam.ft> fit <- sam.fit(nscodData, nscodConf, nscodParameters, silent = TRUE) TMB has received an error from Eigen. The following condition was not met: index >= 0 && index < size() Please check your matrix-vector bounds etc., or run your program through a debugger. Process 24620 stopped

calbertsen commented 1 month ago

It seems the issue is that:

isNAINT(dat.idxCor(f,y))

evaluates to false on macos arm64, but true (as we expect) on all other systems.

calbertsen commented 1 month ago

Can you please: 1) re-install TMB from this fork: https://github.com/calbertsen/adcomp 2) re-install the stock assessment package 3) run the script again to see if it works?

h-du-pontavice commented 1 month ago

It works! I really appreciate your assistance with resolving this issue.