Open bob-carpenter opened 4 years ago
While specializing it would also be useful to consider direct adjoint-Jacobian transpose implementations which can provide significant memory savings for matrix operations.
On Mar 20, 2020, at 6:26 PM, Bob Carpenter notifications@github.com wrote:
Description
There are lots of functions that don't have reverse mode specializations.
Matrix operations
All of these would be very useful, espcially the basic arithmetic of the first five and inverse_spd because it's so important.
add
subtract
minus
elt_multiply
elt_divide
diag_post_multiply
diag_pre_multiply
quad_form_diag
chol2inv
`inverse_spd
All of the constraints would be good to specialize. These get used everywhere (the least important are corr_constrain and cov_matrix_constrain---the cholesky forms are more important for our recommended usages).
lb_constrain ub_constrain lub_constrain corr_constrain cov_matrix_constrain cholesky_factor_constrain cholesky_corr_constrain offset_multiplier_constrain Then there are some reductions:
mean sd variance There are probably more.
Feel free to solve one or more of these in a PR.
Current Version:
v3.1.0
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/math/issues/1787, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALU3FRVL45AGSDJ2BHEXR3RIPURPANCNFSM4LQVVS2Q.
Sounds like there is a lot of performance on the table to grab here! Looking forward to those benchmarks.
Definitely direct adjoint-Jacobian implementations are the way to go to avoid all the virtual function calls and collapse the autodiff graph.
If no one minds, I'll have a shot at some of these over the weekend.
@dpsimpson: All yours.
Description
There are lots of functions that don't have reverse mode specializations.
Matrix operations
All of these would be very useful, espcially the basic arithmetic of the first five and
inverse_spd
because it's so important.add
subtract
minus
elt_multiply
elt_divide
diag_post_multiply
diag_pre_multiply
quad_form_diag
chol2inv
All of the constraints would be good to specialize. These get used everywhere (the least important are
corr_constrain
andcov_matrix_constrain
---the cholesky forms are more important for our recommended usages).lb_constrain
ub_constrain
lub_constrain
corr_constrain
cov_matrix_constrain
cholesky_factor_constrain
cholesky_corr_constrain
offset_multiplier_constrain
Then there are some reductions:
mean
sd
variance
There are probably more.
Feel free to solve one or more of these in a PR.
Current Version:
v3.1.0