Stan/rstan related segfault in package check - is it possible to force rstan to be installed/re-installed from source?

[remlapmot]

I have a package, mrbayes, which uses Stan through the rstan package.

I find on macOS on the GitHub actions runners that R CMD check fails if it uses the binary version of rstan from CRAN. All 4 r-universe macOS checks fail with error messages like:

 *** caught segfault ***
address 0x6574736f682f7265, cause 'invalid permissions'

here is a link to a failing check

https://github.com/r-universe/mrcieu/actions/runs/10455091363/job/28949769156#step:5:898

In my own R CMD check GitHub Action I worked around this by re-installing rstan from source just before running the check, e.g.,

https://github.com/okezie94/mrbayes/blob/3d8c60a6dd28ef4d03c76a01130edbf8a26324fb/.github/workflows/R-CMD-check.yaml#L79-L82

I'm not sure what the difference is between the CRAN macOS setup and the GitHub actions macOS setup which causes this segfault (admittedly it is my strong assumption that this is the cause of this error - I might be wrong).

Any thoughts appreciated.

[jeroen]

Below a crash log, which isn't much more helpful.

I suspect it is related to threading in rstan. rstan builds against RcppParallel, which was recently updated. Perhaps the binary build of rstan from CRAN needs to be rebuilt for the new RcppParallel?

-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Process:               R [8110]
Path:                  /Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/bin/exec/R
Identifier:            R
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        Exited process [8109]
Responsible:           provisioner [549]
User ID:               501

Date/Time:             2024-08-19 21:09:09.9834 +0000
OS Version:            macOS 13.6.9 (22G830)
Report Version:        12
Anonymous UUID:        12FDCAD6-6230-E850-710F-75E7B34DFFDC

Time Awake Since Boot: 1700 seconds

System Integrity Protection: disabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       UNKNOWN_0xD at 0x0000000000000000
Exception Codes:       0x000000000000000d, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   R [8110]

VM Region Info: 0 is not in any region.  Bytes before following region: 4349677568
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      __TEXT                      10342d000-103431000    [   16K] r-x/r-x SM=COW  ...es/bin/exec/R

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib              0x7ff812b24196 __pthread_kill + 10
1   libsystem_pthread.dylib             0x7ff812b5bee6 pthread_kill + 263
2   libsystem_c.dylib                   0x7ff812a45601 raise + 26
3   libsystem_platform.dylib            0x7ff812b895ed _sigtramp + 29
4   ???                                            0x0 ???
5   libR.dylib                             0x1039d1f4e get_package_CEntry_table + 78 (Rdynload.c:1717)
6   libR.dylib                             0x1039d1fd6 R_GetCCallable + 22 (Rdynload.c:1739)
7   mrbayes.so                             0x106d17157 0x106cef000 + 164183
8   mrbayes.so                             0x106d16fda 0x106cef000 + 163802
9   mrbayes.so                             0x106d16f27 0x106cef000 + 163623
10  mrbayes.so                             0x106d161f7 0x106cef000 + 160247
11  mrbayes.so                             0x106d13b22 0x106cef000 + 150306
12  mrbayes.so                             0x106d13a2a 0x106cef000 + 150058
13  mrbayes.so                             0x106d04582 0x106cef000 + 87426
14  Rcpp.so                                0x10394783a class__newInstance(SEXPREC*) + 250 (module.cpp:143)
15  libR.dylib                             0x103a5a8e6 do_External + 294 (dotcode.c:576)
16  libR.dylib                             0x103aaccae bcEval_loop + 41646 (eval.c:8076)
17  libR.dylib                             0x103a94374 bcEval + 628 (eval.c:7524)
18  libR.dylib                             0x103a93a7a Rf_eval + 506 (eval.c:1167)
19  libR.dylib                             0x103a965b9 R_execClosure + 761 (eval.c:2398)
20  libR.dylib                             0x103a956c0 applyClosure_core + 128 (eval.c:2311)
21  libR.dylib                             0x103a93d25 Rf_applyClosure + 24 (eval.c:2333) [inlined]
22  libR.dylib                             0x103a93d25 Rf_eval + 1189 (eval.c:1285)
23  libR.dylib                             0x103a93c5e Rf_eval + 990 (eval.c:1237)
24  libR.dylib                             0x103a98fad do_begin + 429 (eval.c:3010)
25  libR.dylib                             0x103a93c5e Rf_eval + 990 (eval.c:1237)

[jeroen]

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib                 0x18f7cd5f0 __pthread_kill + 8
1   libsystem_pthread.dylib                0x18f805c20 pthread_kill + 288
2   libsystem_c.dylib                      0x18f6dc1e0 raise + 32
3   libsystem_platform.dylib               0x18f836584 _sigtramp + 56
4   libR.dylib                             0x100a52d54 get_package_CEntry_table + 84 (Rdynload.c:1717)
5   libR.dylib                             0x100a52d54 get_package_CEntry_table + 84 (Rdynload.c:1717)
6   libR.dylib                             0x100a52de0 R_GetCCallable + 32 (Rdynload.c:1739)
7   mrbayes.so                             0x1039b8358 char_get_string_elt(SEXPREC*, long) + 36 (routines.h:218) [inlined]
8   mrbayes.so                             0x1039b8358 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> Rcpp::internal::as_string_elt__impl<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>(SEXPREC*, long, Rcpp::traits::integral_constant<bool, false>) + 216 (export.h:36)
9   mrbayes.so                             0x1039b81a0 std::__1::basic_string<Rcpp::traits::char_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>::type, std::__1::char_traits<Rcpp::traits::char_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>::type>, std::__1::allocator<Rcpp::traits::char_type<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>::type>> const Rcpp::internal::as_string_elt<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>(SEXPREC*, long) + 16 (export.h:42) [inlined]
10  mrbayes.so                             0x1039b81a0 void Rcpp::internal::export_range__dispatch<std::__1::__wrap_iter<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>*>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>(SEXPREC*, std::__1::__wrap_iter<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>*>, Rcpp::traits::r_type_string_tag) + 96 (export.h:88)
11  mrbayes.so                             0x1039b80d8 void Rcpp::internal::export_range<std::__1::__wrap_iter<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>*>>(SEXPREC*, std::__1::__wrap_iter<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>*>) + 8 (export.h:94) [inlined]
12  mrbayes.so                             0x1039b80d8 Rcpp::traits::RangeExporter<std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>>::get() + 152 (Exporter.h:47)
13  mrbayes.so                             0x1039b7550 std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> Rcpp::internal::as<std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>>(SEXPREC*, Rcpp::traits::r_type_generic_tag) + 20 (as.h:89) [inlined]
14  mrbayes.so                             0x1039b7550 std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> Rcpp::as<std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>>(SEXPREC*) + 20 (as.h:152) [inlined]
15  mrbayes.so                             0x1039b7550 rstan::io::rlist_ref_var_context::rlist_ref_var_context(SEXPREC*) + 208 (rlist_ref_var_context.hpp:101)
16  mrbayes.so                             0x1039b59f0 rstan::io::rlist_ref_var_context::rlist_ref_var_context(SEXPREC*) + 4 (rlist_ref_var_context.hpp:97) [inlined]
17  mrbayes.so                             0x1039b59f0 rstan::stan_fit<model_mregger_namespace::model_mregger, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u>>>::stan_fit(SEXPREC*, SEXPREC*, SEXPREC*) + 48 (stan_fit.hpp:1021)
18  mrbayes.so                             0x1039b58e8 rstan::stan_fit<model_mregger_namespace::model_mregger, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u>>>::stan_fit(SEXPREC*, SEXPREC*, SEXPREC*) + 4 (stan_fit.hpp:1031) [inlined]
19  mrbayes.so                             0x1039b58e8 rstan::stan_fit<model_mregger_namespace::model_mregger, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u>>>* Rcpp::Constructor<rstan::stan_fit<model_mregger_namespace::model_mregger, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u>>>, SEXPREC*, SEXPREC*, SEXPREC*>::get_new_impl<0, 1, 2>(SEXPREC**, int, Rcpp::traits::index_sequence<0, 1, 2>) + 24 (Module.h:213) [inlined]
20  mrbayes.so                             0x1039b58e8 Rcpp::Constructor<rstan::stan_fit<model_mregger_namespace::model_mregger, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u>>>, SEXPREC*, SEXPREC*, SEXPREC*>::get_new(SEXPREC**, int) + 40 (Module.h:203)
21  mrbayes.so                             0x1039a60d4 Rcpp::class_<rstan::stan_fit<model_mregger_namespace::model_mregger, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u>>>>::newInstance(SEXPREC**, int) + 404 (class.h:143)
22  Rcpp.so                                0x100485254 class__newInstance(SEXPREC*) + 212 (module.cpp:143)
23  libR.dylib                             0x100aced6c do_External + 300 (dotcode.c:576)
24  libR.dylib                             0x100b162b0 bcEval_loop + 37744 (eval.c:8076)
25  libR.dylib                             0x100aff56c bcEval + 684 (eval.c:7524)
26  libR.dylib                             0x100afec6c Rf_eval + 556 (eval.c:1167)

[jeroen]

Maybe you could try to add a copy of rstan to your own universe to see if the CRAN binary really is the problem?

[remlapmot]

Thanks for the investigation.

Good idea.

I've added rstan from the GitHub CRAN mirror https://github.com/cran/rstan because the repo for rstan seems to be behind the version on CRAN https://github.com/stan-dev/rstan/tree/develop/rstan/rstan

[jeroen]

Looks like this hasn't fixed the issue :(