search

pmelchior / scarlet2

Scarlet, all new and shiny
MIT License
15 stars 4 forks source link

diagonalized fast Fisher matrix #88

Open SampsonML opened 1 week ago

SampsonML commented 1 week ago

Move the implementation of the Hessian vector product approximation of the Fisher matrix into a standalone, runable location (maybe utils.py) https://github.com/pmelchior/scarlet2/blob/9360b42c50ee20e364b66916840e50685bde4de7/scarlet2/plot.py#L555

This will include the log_like implementation (https://github.com/pmelchior/scarlet2/blob/9360b42c50ee20e364b66916840e50685bde4de7/scarlet2/plot.py#L534)

as well as the neural gradient (https://github.com/pmelchior/scarlet2/blob/9360b42c50ee20e364b66916840e50685bde4de7/scarlet2/plot.py#L526)

pmelchior commented 1 week ago

In principle, yes. Just to make this more clear. I suggest moving the diagonalize Hessian/Rademacher methods to a new file utils.py: https://github.com/pmelchior/scarlet2/blob/9360b42c50ee20e364b66916840e50685bde4de7/scarlet2/plot.py#L525-L568

But with a few modifications: log_like should not be copied. In fact, log_like shouldn't have been implemented here either because obs has that method already. This code doesn't account for the rendering operator, so it's not quite right anyway.

I also want to generalize this such that you can provide any parameter and get its Hessian. plot can then call the functions from there. In addition, Scene should get a new method .hessian(observations, parameter), which can construct the same loss function that we're using during fit.