I wrote a package to pre-compile CmdStan models inside other R packages. The project is not public yet, but after it goes through my company's open-source approval process (hopefully within 2 weeks), it will be available at https://github.com/wlandau/instantiate.
The goal of instantiate is to do for cmdstanr what rstantools does for rstan: in a statistical modeling R package, compile the Stan models only once and only at installation time. Besides the obvious efficiency gains, this approach will improve the tooling in large companies with centrally maintained R installations: the dedicated devops people who install all the R packages can save end-users the trouble of debugging just-in-time compilation issues.
How it works
instantiate hard-codes the value of $CMDSTAN when it installs so that downstream packages do not have to worry about finding the path to CmdStan or setting environment variables.
For a package with models in inst/stan/, instantiate writes configure and configure.win to make sure install.packages() finds CmdStan and compiles all the Stan models in-place.
The documentation describes the Additional_repositories: https://mc-stan.org/r-packages/ approach @jgabry mentioned in https://github.com/stan-dev/cmdstanr/issues/738. (And I agree with @danielinteractive, it would be awesome if cmdstanr itself made it to CRAN!)
Issue
On my M2 MacBook, the compiler flags set by cmdstanr::cmdstan_model() are different within install.packages() vs from outside install.packages(). In the former case, my machine automatically sets -falign-functions=64, which disagrees with the alignment in the CmdStan PCH file and causes compilation to fail. How do I prevent this flag from being set? Alternatively, if there is a way to get the value of -falign-functions that CmdStan sets, I can set it manually on my end.
Reproducible example
Here is an oversimplified version of the package setup process I implemented for instantiate:
install.packages(pkgs = package, lib = library, repos = NULL, type = "source") fails with the following:
* installing *source* package ‘file16fef785b3954’ ...
** using staged installation
Running make \
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpzheQTt/model-54d27d0dfd41
--- Translating Stan model to C++ code ---
bin/stanc --o=/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpzheQTt/model-54d27d0dfd41.hpp /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpzheQTt/model-54d27d0dfd41.stan
--- Compiling, linking C++ code ---
clang++ -arch arm64 -std=gnu++17 -falign-functions=64 -Wall -g -O2 -std=c++1y -Wno-unknown-warning-option -Wno-tautological-compare -Wno-sign-compare -D_REENTRANT -Wno-ignored-attributes -I stan/lib/stan_math/lib/tbb_2020.3/include -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/R/arm64/include -DBOOST_DISABLE_ASSERTS -c -include-pch stan/src/stan/model/model_header.hpp.gch -x c++ -o /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpzheQTt/model-54d27d0dfd41.o /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpzheQTt/model-54d27d0dfd41.hpp
error: Default alignment for functions differs in PCH file vs. current file
error: Default alignment for functions differs in PCH file vs. current file
1 error generated.
make: *** [/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpzheQTt/model-54d27d0dfd41] Error 1
1 error generated.
make: *** [/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpzheQTt/model-54d27d0dfd41] Error 1
Error: An error occured during compilation! See the message above for more information.
In addition: Warning message:
CmdStan's precompiled header (PCH) files may need to be rebuilt.
If your model failed to compile please run rebuild_cmdstan().
If the issue persists please open a bug report.
Execution halted
ERROR: configuration failed for package ‘file16fef785b3954’
* removing ‘/private/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpcnkKva/file16fef2d10f787/file16fef785b3954’
Warning in install.packages :
installation of package ‘/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T//RtmpcnkKva/file16fef785b3954’ had non-zero exit status
So I tried manually setting -falign-functions=1, which works on my machine, but is just a temporary hack from a tool development point of view.
After that, install.packages(pkgs = package, lib = library, repos = NULL, type = "source") still sets -falign-functions=64 but then overrides it -falign-functions=1. Compilation succeeds:
* installing *source* package ‘file16fef785b3954’ ...
** using staged installation
Running make \
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af
--- Translating Stan model to C++ code ---
bin/stanc --o=/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.stan
--- Compiling, linking C++ code ---
clang++ -arch arm64 -std=gnu++17 -falign-functions=64 -Wall -g -O2 -falign-functions=1 -std=c++1y -Wno-unknown-warning-option -Wno-tautological-compare -Wno-sign-compare -D_REENTRANT -Wno-ignored-attributes -I stan/lib/stan_math/lib/tbb_2020.3/include -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/R/arm64/include -DBOOST_DISABLE_ASSERTS -c -include-pch stan/src/stan/model/model_header.hpp.gch -x c++ -o /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.o /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:38:11: warning: variable 'pos__' set but not used [-Wunused-but-set-variable]
int pos__ = std::numeric_limits<int>::min();
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:38:11: warning: variable 'pos__' set but not used [-Wunused-but-set-variable]
int pos__ = std::numeric_limits<int>::min();
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:170:11: warning: variable 'pos__' set but not used [-Wunused-but-set-variable]
int pos__ = std::numeric_limits<int>::min();
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:194:11: warning: variable 'pos__' set but not used [-Wunused-but-set-variable]
int pos__ = std::numeric_limits<int>::min();
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:170:11: warning: variable 'pos__' set but not used [-Wunused-but-set-variable]
int pos__ = std::numeric_limits<int>::min();
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:194:11: warning: variable 'pos__' set but not used [-Wunused-but-set-variable]
int pos__ = std::numeric_limits<int>::min();
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:10:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun.hpp:55:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun/elt_multiply.hpp:9:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun/multiply.hpp:7:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun.hpp:124:
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:192:12: warning: unused variable 'pre_mult' [-Wunused-variable]
auto pre_mult = a2 * pow(1 - z, -1 - a2);
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:307:20: note: in instantiation of function template specialization 'stan::math::internal::grad_2F1_impl<true, true, true, true, double, double, double, double, double, std::tuple<double, double, double, double>>' requested here
return internal::grad_2F1_impl<true, true, true, true>(a1, a2, b1, z,
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:10:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun.hpp:55:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun/elt_multiply.hpp:9:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun/multiply.hpp:7:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun.hpp:124:
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:192:12: warning: unused variable 'pre_mult' [-Wunused-variable]
auto pre_mult = a2 * pow(1 - z, -1 - a2);
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:307:20: note: in instantiation of function template specialization 'stan::math::internal::grad_2F1_impl<true, true, true, true, double, double, double, double, double, std::tuple<double, double, double, double>>' requested here
return internal::grad_2F1_impl<true, true, true, true>(a1, a2, b1, z,
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:192:12: warning: unused variable 'pre_mult' [-Wunused-variable]
auto pre_mult = a2 * pow(1 - z, -1 - a2);
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:307:20: note: in instantiation of function template specialization 'stan::math::internal::grad_2F1_impl<true, true, true, true, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, std::tuple<stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>>>' requested here
return internal::grad_2F1_impl<true, true, true, true>(a1, a2, b1, z,
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:192:12: warning: unused variable 'pre_mult' [-Wunused-variable]
auto pre_mult = a2 * pow(1 - z, -1 - a2);
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:307:20: note: in instantiation of function template specialization 'stan::math::internal::grad_2F1_impl<true, true, true, true, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, std::tuple<stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>>>' requested here
return internal::grad_2F1_impl<true, true, true, true>(a1, a2, b1, z,
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:8:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core.hpp:29:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_divide_equal.hpp:5:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_division.hpp:13:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_multiplication.hpp:7:
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_subtraction.hpp:104:21: warning: lambda capture 'a' is not used [-Wunused-lambda-capture]
[bvi = b.vi_, a](const auto& vi) mutable { bvi->adj_ -= vi.adj_; });
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:191:50: note: in instantiation of function template specialization 'stan::math::operator-<int, nullptr>' requested here
auto hyper2 = hypergeometric_2F1(1 + a2, 1 - a1 + b1, 1 + b1, z_euler);
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:307:20: note: in instantiation of function template specialization 'stan::math::internal::grad_2F1_impl<true, true, true, true, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, std::tuple<stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>>>' requested here
return internal::grad_2F1_impl<true, true, true, true>(a1, a2, b1, z,
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:8:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core.hpp:29:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_divide_equal.hpp:5:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_division.hpp:13:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_multiplication.hpp:7:
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_subtraction.hpp:104:21: warning: lambda capture 'a' is not used [-Wunused-lambda-capture]
[bvi = b.vi_, a](const auto& vi) mutable { bvi->adj_ -= vi.adj_; });
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:191:50: note: in instantiation of function template specialization 'stan::math::operator-<int, nullptr>' requested here
auto hyper2 = hypergeometric_2F1(1 + a2, 1 - a1 + b1, 1 + b1, z_euler);
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:307:20: note: in instantiation of function template specialization 'stan::math::internal::grad_2F1_impl<true, true, true, true, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, std::tuple<stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>>>' requested here
return internal::grad_2F1_impl<true, true, true, true>(a1, a2, b1, z,
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:8:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core.hpp:29:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_divide_equal.hpp:5:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_division.hpp:13:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_multiplication.hpp:7:
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_subtraction.hpp:104:21: warning: lambda capture 'a' is not used [-Wunused-lambda-capture]
[bvi = b.vi_, a](const auto& vi) mutable { bvi->adj_ -= vi.adj_; });
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:208:38: note: in instantiation of function template specialization 'stan::math::operator-<double, nullptr>' requested here
auto pre_mult_ab = inv(pow(1.0 - z, a2));
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:307:20: note: in instantiation of function template specialization 'stan::math::internal::grad_2F1_impl<true, true, true, true, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, std::tuple<stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>>>' requested here
return internal::grad_2F1_impl<true, true, true, true>(a1, a2, b1, z,
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:8:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core.hpp:29:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_divide_equal.hpp:5:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_division.hpp:13:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_multiplication.hpp:7:
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/core/operator_subtraction.hpp:104:21: warning: lambda capture 'a' is not used [-Wunused-lambda-capture]
[bvi = b.vi_, a](const auto& vi) mutable { bvi->adj_ -= vi.adj_; });
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:208:38: note: in instantiation of function template specialization 'stan::math::operator-<double, nullptr>' requested here
auto pre_mult_ab = inv(pow(1.0 - z, a2));
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/fun/grad_2F1.hpp:307:20: note: in instantiation of function template specialization 'stan::math::internal::grad_2F1_impl<true, true, true, true, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, std::tuple<stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>, stan::math::var_value<double>>>' requested here
return internal::grad_2F1_impl<true, true, true, true>(a1, a2, b1, z,
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:10:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun.hpp:198:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/functor.hpp:15:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/functor/integrate_ode_rk45.hpp:6:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/functor/ode_rk45.hpp:9:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/numeric/odeint.hpp:76:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/numeric/odeint/integrate/observer_collection.hpp:23:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/function.hpp:30:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/function/detail/prologue.hpp:17:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/function/function_base.hpp:21:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/type_index.hpp:29:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/type_index/stl_type_index.hpp:47:
/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:132:33: warning: 'unary_function<const std::error_category *, unsigned long>' is deprecated [-Wdeprecated-declarations]
struct hash_base : std::unary_function<T, std::size_t> {};
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:692:18: note: in inIn file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:10:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun.hpp:198:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/functor.hpp:15:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/functor/integrate_ode_rk45.hpp:6:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/prim/functor/ode_rk45.hpp:9:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/numeric/odeint.hpp:76:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/numeric/odeint/integrate/observer_collection.hpp:23:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/function.hpp:30:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/function/detail/prologue.hpp:17:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/function/function_base.hpp:21:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/type_index.hpp:29:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/type_index/stl_type_index.hpp:47:
/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:132:33: warning: 'unary_function<const std::error_category *, unsigned long>' is deprecated [-Wdeprecated-declarations]
struct hash_base : std::unary_function<T, std::size_t> {};
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:692:18: note: in in
stantiation of template class 'boost::hash_detail::hash_base<const std::error_category *>' requested here
: public boost::hash_detail::hash_base<T*>
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:420:24: note: in instantiation of template class 'boost::hash<const std::error_category *>' requested here
boost::hash<T> hasher;
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:551:9: note: in instantiation of function template specialization 'boost::hash_combine<const std::error_category *>' requested here
hash_combine(seed, &v.category());
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__functional/unary_function.h:23:29: note: 'unary_function<const std::error_category *, unsigned long>' has been explicitly marked deprecated here
struct _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX11 unary_function
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:825:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX11'
# define _LIBCPP_DEPRECATED_IN_CXX11 _LIBCPP_DEPRECATED
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:810:49: note: expanded from macro '_LIBCPP_DEPRECATED'
# define _LIBCPP_DEPRECATED __attribute__((deprecated))
^
stantiation of template class 'boost::hash_detail::hash_base<const std::error_category *>' requested here
: public boost::hash_detail::hash_base<T*>
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:420:24: note: in instantiation of template class 'boost::hash<const std::error_category *>' requested here
boost::hash<T> hasher;
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/boost_1.78.0/boost/container_hash/hash.hpp:551:9: note: in instantiation of function template specialization 'boost::hash_combine<const std::error_category *>' requested here
hash_combine(seed, &v.category());
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__functional/unary_function.h:23:29: note: 'unary_function<const std::error_category *, unsigned long>' has been explicitly marked deprecated here
struct _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX11 unary_function
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:825:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX11'
# define _LIBCPP_DEPRECATED_IN_CXX11 _LIBCPP_DEPRECATED
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/__config:810:49: note: expanded from macro '_LIBCPP_DEPRECATED'
# define _LIBCPP_DEPRECATED __attribute__((deprecated))
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:10:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun.hpp:100:
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun/lb_constrain.hpp:114:33: warning: lambda capture 'lp' is not used [-Wunused-lambda-capture]
[lp, arena_lb = var(lb)](auto& vi) mutable {
^
In file included from /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:1:
In file included from /Users/CENSORED/cmdstan/stan/src/stan/model/model_header.hpp:4:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math.hpp:19:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev.hpp:10:
In file included from /Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun.hpp:100:
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun/lb_constrain.hpp:114:33: warning: lambda capture 'lp' is not used [-Wunused-lambda-capture]
[lp, arena_lb = var(lb)](auto& vi) mutable {
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun/lub_constrain.hpp:115:12: note: in instantiation of function template specialization 'stan::math::lb_constrain<stan::math::var_value<double>, int, nullptr, nullptr>' requested here
return lb_constrain(identity_constrain(x, ub), lb, lp);
^
/Users/CENSORED/cmdstan/stan/src/stan/io/deserializer.hpp:441:26: note: in instantiation of function template specialization 'stan::math::lub_constrain<stan::math::var_value<double>, int, int, nullptr, nullptr>' requested here
return stan::math::lub_constrain(this->read<Ret>(sizes...), lb, ub, lp);
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:94:29: note: in instantiation of function template specialization 'stan::io::deserializer<stan::math::var_value<double>>::read_constrain_lub<stan::math::var_value<double>, false, int, int, stan::math::var_value<double>>' requested here
theta = in__.template read_constrain_lub<local_scalar_t__,
^
/Users/CENSORED/cmdstan/stan/lib/stan_math/stan/math/rev/fun/lub_constrain.hpp:115:12: note: in instantiation of function template specialization 'stan::math::lb_constrain<stan::math::var_value<double>, int, nullptr, nullptr>' requested here
return lb_constrain(identity_constrain(x, ub), lb, lp);
^
/Users/CENSORED/cmdstan/stan/src/stan/io/deserializer.hpp:441:26: note: in instantiation of function template specialization 'stan::math::lub_constrain<stan::math::var_value<double>, int, int, nullptr, nullptr>' requested here
return stan::math::lub_constrain(this->read<Ret>(sizes...), lb, ub, lp);
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:94:29: note: in instantiation of function template specialization 'stan::io::deserializer<stan::math::var_value<double>>::read_constrain_lub<stan::math::var_value<double>, false, int, int, stan::math::var_value<double>>' requested here
theta = in__.template read_constrain_lub<local_scalar_t__,
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:279:12: note: in instantiation of function template specialization 'model_58822746f8af_model_namespace::model_58822746f8af_model::log_prob_impl<false, false, Eigen::Matrix<stan::math::var_value<double>, -1, 1, 0>, Eigen::Matrix<int, -1, 1, 0>, nullptr, nullptr>' requested here
return log_prob_impl<propto__, jacobian__>(params_r, params_i, pstream);
^
/Users/CENSORED/cmdstan/stan/src/stan/model/model_base_crtp.hpp:98:50: note: in instantiation of function template specialization 'model_58822746f8af_model_namespace::model_58822746f8af_model::log_prob<false, false, stan::math::var_value<double>>' requested here
return static_cast<const M*>(this)->template log_prob<false, false>(theta,
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:20:3: note: in instantiation of member function 'stan::model::model_base_crtp<model_58822746f8af_model_namespace::model_58822746f8af_model>::log_prob' requested here
~model_58822746f8af_model() {}
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:279:12: note: in instantiation of function template specialization 'model_58822746f8af_model_namespace::model_58822746f8af_model::log_prob_impl<false, false, Eigen::Matrix<stan::math::var_value<double>, -1, 1, 0>, Eigen::Matrix<int, -1, 1, 0>, nullptr, nullptr>' requested here
return log_prob_impl<propto__, jacobian__>(params_r, params_i, pstream);
^
/Users/CENSORED/cmdstan/stan/src/stan/model/model_base_crtp.hpp:98:50: note: in instantiation of function template specialization 'model_58822746f8af_model_namespace::model_58822746f8af_model::log_prob<false, false, stan::math::var_value<double>>' requested here
return static_cast<const M*>(this)->template log_prob<false, false>(theta,
^
/var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.hpp:20:3: note: in instantiation of member function 'stan::model::model_base_crtp<model_58822746f8af_model_namespace::model_58822746f8af_model>::log_prob' requested here
~model_58822746f8af_model() {}
^
9 warnings generated.
9 warnings generated.
clang++ -arch arm64 -std=gnu++17 -falign-functions=64 -Wall -g -O2 -falign-functions=1 -std=c++1y -Wno-unknown-warning-option -Wno-tautological-compare -Wno-sign-compare -D_REENTRANT -Wno-ignored-attributes -I stan/lib/stan_math/lib/tbb_2020.3/include -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -I/opt/R/arm64/include -DBOOST_DISABLE_ASSERTS -L/opt/R/arm64/lib -Wl,-L,"/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/tbb" /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.o src/cmdstan/main.o -Wl,-L,"/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/Users/CENSORED/cmdstan/stan/lib/stan_math/lib/tbb" stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a stan/lib/stan_math/lib/tbb/libtbb.dylib stan/lib/stan_math/lib/tbb/libtbbmalloc.dylib stan/lib/stan_math/lib/tbb/libtbbmalloc_proxy.dylib -o /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af
rm -f /var/folders/4v/vh7xp8553lsbl49svl48g7p00000gp/T/RtmpXxOUHv/model-58822746f8af.o
data {
int<lower=0> N;
array[N] int<lower=0,upper=1> y;
}
parameters {
real<lower=0,upper=1> theta;
}
model {
theta ~ beta(1,1); // uniform prior on interval 0,1
y ~ bernoulli(theta);
}
** inst
** help
No man pages found in package ‘file16fef785b3954’
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (file16fef785b3954)
Background
I wrote a package to pre-compile CmdStan models inside other R packages. The project is not public yet, but after it goes through my company's open-source approval process (hopefully within 2 weeks), it will be available at https://github.com/wlandau/instantiate.
The goal of
instantiate
is to do forcmdstanr
whatrstantools
does forrstan
: in a statistical modeling R package, compile the Stan models only once and only at installation time. Besides the obvious efficiency gains, this approach will improve the tooling in large companies with centrally maintained R installations: the dedicated devops people who install all the R packages can save end-users the trouble of debugging just-in-time compilation issues.How it works
instantiate
hard-codes the value of$CMDSTAN
when it installs so that downstream packages do not have to worry about finding the path to CmdStan or setting environment variables.inst/stan/
,instantiate
writesconfigure
andconfigure.win
to make sureinstall.packages()
finds CmdStan and compiles all the Stan models in-place.Additional_repositories: https://mc-stan.org/r-packages/
approach @jgabry mentioned in https://github.com/stan-dev/cmdstanr/issues/738. (And I agree with @danielinteractive, it would be awesome ifcmdstanr
itself made it to CRAN!)Issue
On my M2 MacBook, the compiler flags set by
cmdstanr::cmdstan_model()
are different withininstall.packages()
vs from outsideinstall.packages()
. In the former case, my machine automatically sets-falign-functions=64
, which disagrees with the alignment in the CmdStan PCH file and causes compilation to fail. How do I prevent this flag from being set? Alternatively, if there is a way to get the value of-falign-functions
that CmdStan sets, I can set it manually on my end.Reproducible example
Here is an oversimplified version of the package setup process I implemented for
instantiate
:install.packages(pkgs = package, lib = library, repos = NULL, type = "source")
fails with the following:So I tried manually setting
-falign-functions=1
, which works on my machine, but is just a temporary hack from a tool development point of view.After that,
install.packages(pkgs = package, lib = library, repos = NULL, type = "source")
still sets-falign-functions=64
but then overrides it-falign-functions=1
. Compilation succeeds:Session info