Sparse matrix in R incompatible with STAN

[leerichardson]

Summary:

Integrate R's sparse matrix class from the Matrix library with STAN.

Description:

When I use the Sparse Matrix class in R, provided by the Matrix package, my data-set is ~ 10 Mb. But when I call an RStan program I need to convert my matrix using as.matrix() base class in order for the function to work.

Unfortunately, my matrix is very sparse so my data-set goes from 10 Mb to 2 GB when I call as.matrix, and my R session crashes due to memory error.

My question is: can STAN use the SparseMatrix class from R in its computation? If so, how would I go about using this?

Reproducible Steps:

Current Output:

If applicable, any relevant output from RStan.

Expected Output:

If applicable, the output you expected from RStan.

RStan Version:

2.21.1

R Version:

4.02

Operating System:

Your operating system (e.g., Ubuntu 20.04

[jgabry]

My question is: can STAN use the SparseMatrix class from R in its computation?

No, unfortunately Stan doesn't know how to deal with any of R's special classes (or Python's, etc.), but I think some people are working on introducing sparse matrix types into Stan directly. Currently I think the only special sparse matrix functionality in Stan is sparse matrix times dense vector. If that's the use case you have then you can use the csr_matrix_times_vector() function in Stan and use rstan::extract_sparse_parts() to extract and pass the necessary data to Stan:

• https://mc-stan.org/docs/2_25/functions-reference/sparse-matrix-arithmetic.html

• https://mc-stan.org/rstan/reference/extract_sparse_parts.html

[leerichardson]

Thank you so much! Really helpful.

One question: what do you typically recommend folks do in these situations? Should I try to use the c++ version? Get a bigger computer?? Just curious, thank you!