search

SciML / SciMLSensitivity.jl

A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.
https://docs.sciml.ai/SciMLSensitivity/stable/
Other
322 stars 69 forks source link

Fisher Information Matrix computation #951

Open TorkelE opened 6 months ago

TorkelE commented 6 months ago

The fisher information matrix (basically the hessian with respect to the cost function) is frequently used in systems biology for practical identifiability analysis. I think this would be useful to have access to for various inverse problems. In practise this should be possible to compute for most SciML inverse problems using SciMLSensitivity. I tried having a go at creating such a function myself, but didn't I didn't really manage to figure out the right sub-functions.

Would it make sense to have this? In practise, we would basically want a fisher_information_matrix function, which when applied to a function and a input value, gives the FIM. For OptimizationFunctions, given stuff is already there, this should pretty much just be extracting the right fields.

Does this make sense, also, where should the function be put?

ChrisRackauckas commented 6 months ago

Forward over forward/reverse just works, so FowrardDiff.hessian and all work, along with functionality like https://docs.sciml.ai/SciMLSensitivity/dev/manual/direct_adjoint_sensitivities/#Second-Order-Adjoint-Sensitivities

TorkelE commented 6 months ago

Would it make sense to create some pretty wrapper functions for this functionality?

ChrisRackauckas commented 6 months ago

Like what? The link that I posted is kind of the pretty wrapper function. Would you want another step above that directly named fisher_information_matrix? That would be fine and I'd accept it.

TorkelE commented 6 months ago

Sounds good. I will make a tutorial on practical identifiability in the relatively soon future. I will start of using this, and then when I am done put the wrapper in. Should make it a bit more pleasant for systems biologists working with it.