search

pbrod / numdifftools

Solve automatic numerical differentiation problems in one or more variables.
BSD 3-Clause "New" or "Revised" License
255 stars 44 forks source link

vector-valued Hessian #47

Open naodell opened 5 years ago

naodell commented 5 years ago

I use the Hessian tool to calculate the covariance matrix of a cost function (NLL). The NLL is calculated for multiple categories of data and then combined in a single number currently, but I would like to be able to factorize the Hessian of the NLL to account for the variance in each of the data categories. I know I could carry it out in each category independently, but I would then lose sensitivity to the parameter's correlations between the individual categories of data.

I would be willing to try to carry out the implementation myself if you could provide some guidance on how to get started.

Thanks!

pbrod commented 3 years ago

Isn't that what the Hessian wrapper class do already? The Hessian wrapper computes the Hessian matrix of the scalar function f with respect to a vector constructed from all variables (data categories) found in f:

<img src= "https://render.githubusercontent.com/render/math?math=%5Cdisplaystyle+%5Cbegin%7Balign%2A%7D%0A%28%5CHessian+f%29_%7Bij%7D+%26%5Cequiv+%5Cfrac%7B%5Cpartial%5E%7B2%7D+f%7D%7B%5Cpartial+x_%7Bi%7D+%5Cpartial+x_%7Bj%7D+%7D%0A%5Cend%7Balign%2A%7D%0A" alt="\begin{align} (\Hessian f){ij} &\equiv \frac{\partial^{2} f}{\partial x{i} \partial x_{j} } \end{align} ">