Factorisation Checklist

[willtebbutt]

We currently only have partial support for the Cholesky factorisation, and no support for other factorisations. A (non-exhaustive) checklist of basics is:

• [x] chol
• [ ] qr
• [ ] lu
• [ ] svd
• [ ] eig

Its not clear to me whether we technically need to cover the following given the former, but I'll add them anyway:

• [ ] cholfact
• [ ] qrfact
• [ ] lufact
• [ ] svdfact
• [ ] eigfact

I haven't discussed the various specialisations for structured matrices that are possible (e.g. banded matrices are closed under chol). These will be discussed at a later point.

Relevant literature: @article{seeger2017auto, title={Auto-differentiating linear algebra}, author={Seeger, Matthias and Hetzel, Asmus and Dai, Zhenwen and Lawrence, Neil D}, journal={arXiv preprint arXiv:1710.08717}, year={2017} } @article{murray2016differentiation, title={Differentiation of the Cholesky decomposition}, author={Murray, Iain}, journal={arXiv preprint arXiv:1602.07527}, year={2016} } @article{giles2008extended, title={An extended collection of matrix derivative results for forward and reverse mode automatic differentiation}, author={Giles, Mike}, year={2008}, publisher={Unspecified} }

[Ken-B]

Another recent relevant article from NIPS 2018: @incollection{LaueMG2018, title = {Computing Higher Order Derivatives of Matrix and Tensor Expressions}, author = {Laue, S\"{o}ren and Mitterreiter, Matthias and Giesen, Joachim}, booktitle = {Advances in Neural Information Processing Systems (NIPS)}, year = {2018} }