optimize _draw_auto_regression_correlated_np

[mathause]

The function _draw_auto_regression_correlated_np can be optimized to run faster - there are some fun optimizations that can be made.

• [ ] (1) Instead of using np.sum(a * b) we can use np.einsum which should be about twice as fast:

ar = np.einsum("ijk,jk->ik", out[:, t - ar_lags, :], coefs)

https://github.com/MESMER-group/mesmer/blob/0c5b26569c8f0929d20c84448e7c8dc3635aff25/mesmer/core/auto_regression.py#L72

• [ ] (2) (Ab-)use innovations as out. I think we don't actually need to create an out variable. This will avoid using twice the memory and should be a bit faster. To make it less confusing we can just do a shallow copy of the innovations:

out = innovations # copy-by-reference: use innovations as out param

https://github.com/MESMER-group/mesmer/blob/0c5b26569c8f0929d20c84448e7c8dc3635aff25/mesmer/core/auto_regression.py#L69

• [ ] (3) Add the intercept to the innovations. The intercept is added in each pass - this may be suboptimal. However, we need to be careful about the buffer period.

https://github.com/MESMER-group/mesmer/blob/0c5b26569c8f0929d20c84448e7c8dc3635aff25/mesmer/core/auto_regression.py#L74

---

Advanced optimizations:

• [ ] (4) It may be possible to accelerate the for loop using numba.
• [ ] (5) The one dimensional case (n_coefs=1 could be sped up by (i) using out[:, t - ar_lags] @ ar_coefs (even faster than einsum) and np.random.normal instead of np.random.multivariate_normal. However, I now unified the two function deliberately to make it easier to understand.

[mathause]

Removing from the milestone again. This remains a not-critical good first issue.

[mathause]

I suspect that (2) & (3) are mutually exclusive.

[mathause]

For (5) this could be dispatched in draw_auto_regression_correlated (which still has to be written)

[veni-vidi-vici-dormivi]

For (5) we could a separate function for draw innovations, which would make the whole thing more intuitive. However, as mentioned, this was painstakingly unified, I'll look through the whole autoregression part, maybe I can come up with something nice, but as you said, it's not critical.