search

SciML / Optimization.jl

Mathematical Optimization in Julia. Local, global, gradient-based and derivative-free. Linear, Quadratic, Convex, Mixed-Integer, and Nonlinear Optimization in one simple, fast, and differentiable interface.
https://docs.sciml.ai/Optimization/stable/
MIT License
687 stars 75 forks source link

Simple interface for recording optimization trajectory #631

Open TorkelE opened 7 months ago

TorkelE commented 7 months ago

It would be useful to have a good interface for recording the path of all evaluated us and their fitness values. I have tried to make this happen, but it requires hacky solutions (e.g. creating a custom optimization function which also saved the evaluations).

PEtab.jl supports this, and it enables to e.g. make various plots that evaluates how successful the optimisation was: https://sebapersson.github.io/PEtab.jl/stable/optimisation_output_plotting/

Something like 

opt_prob = OptimizationProblem(...)
sol = solve(opt_prob, alg; save_trace=true)

and then a vector of all evaluated us and their cost function value is stored ina field in sol.

Vaibhavdixit02 commented 3 months ago

https://docs.sciml.ai/Optimization/stable/API/optimization_state/#optstate allows access to those values in the callback now

TorkelE commented 3 months ago

Would it be possible to have an option for getting this auto-recorded via an argument, or maybe have a prepared callback that makes this possible without setting up your own system?

Ideally, having a vector of OptimizationState stored in the OptimizationSolution would be really neat (and if one is afraid that that takes lots of space, one could have a kwarg to turn that ON/OFF).

Vaibhavdixit02 commented 3 months ago

Would it be possible to have an option for getting this auto-recorded via an argument,

Would you prefer to evaluate it after running the optimization? You already have access to it while running the optimization in case you want to use it for something - including storing it in an array. Storing it and returning it at the end is not necessarily useful right now so shouldn't be the default. Some of the solvers do have the option to store a trace over the iterations allowing even much more than just iterations and the objective, so that's already possible to set on/off based on kwargs and accessible through the original field of the OptimizationSolution.