Higher-order multivariate derivative

soerenwolfers commented 5 years ago

Assume f is a function that consumes numpy arrays of shape N x d and returns arrays of the form N (i.e., it is a map from R^d to R and allows for vectorized calls).

What is the most elegant way to compute $\partial^{n0}{0}\dots\partial^{nd}{d}f$ using numdifftools, if possible in a vectorized fashion as well?

pbrod commented 3 years ago

Currently there is no easy way to calculate the general multivariate partial derivative like this $\begin{align*} \frac{\partial^{n_0}}{\partial x_0^{n_0}}\dots\frac{\partial^{n_d} }{\partial x_d^{n_d}} f(x_0, \ldots, x_d) \end{align*}$

with numdifftools. However, for n=1 and n=2 you can easily calculate partial derivatives $\begin{align*} \frac{\partial^{n} }{\partial x_0^{n}} f(x_0, x_1,\ldots, x_d), \frac{\partial^{n} }{\partial x_1^{n}} f(x_0, x_1,\ldots, x_d), \ldots, \frac{\partial^{n} }{\partial x_d^{n}} f(x_0, x_1,\ldots, x_d) \end{align*}$ using the numdifftools wrappers Jacobian, Hessdiag and Hessian.

An easy fix to get high order partial derivatives from numdifftools is to modify the Jacobian by removing the restriction that n must be one. Then you can get reasonable reliable partial derivaties $\begin{align*} \frac{\partial^{n} }{\partial x_0^{n}} f(x_0, x_1,\ldots, x_d), \frac{\partial^{n} }{\partial x_1^{n}} f(x_0, x_1,\ldots, x_d), \ldots, \frac{\partial^{n} }{\partial x_d^{n}} f(x_0, x_1,\ldots, x_d) \end{align*}$ up to n equal about 10 depending on the function you are differentiating.

Nickleaton commented 3 years ago

This would be a very useful enhancement.

Nickleaton commented 3 years ago

Looking at some bits of sklearn, there is the idea of a power used to specify terms in a multivariate polynomial

For example. (2,1,0) means x[0]^2 x[1]^1 x[2]^0 = x[0]^2 * x[1].

See PolynomialFeatures from sklearn.preprocessing

Passing in powers in this way could be used to specify that you want d^3X/dx^2dy for example

pbrod / numdifftools

Higher-order multivariate derivative #48