What is PolyOpt()?

[stephans3]

As stated in the title I am not able to find any suitable explanation about PolyOpt(). It is used in several examples in the Sensitivity + DiffEqFlux docs but it is not explained there.

Maybe it is BFGS as stated here #335 ? It would be very helpful to have at least a tiny documentation about it.

Somehow related but off-topic: The list of "packages" that are necessary to run an example like Optimization of Ordinary Differential Equations is growing rapidly. Beside the standard packages (DifferentialEquations, Plots, Zygote), now we have to include Optimization, OptimizationPolyalgorithms, OptimizationOptimJL and SciMLSensitivity. I know that Optimization as a general interface for optimization packages is useful but I find it really hard to say what are the purposes of each of these "packages".

A couple of months ago we only needed to include

• DifferentialEquations / OrdinaryDiffEq = ODE Solver
• Flux = ML backend
• DiffEqFlux = "bridge between ODE and ML"

to solve such examples.

I really like the SciML ecosystem but I find it in some points quite confusing at the moment.

[ChrisRackauckas]

It would be very helpful to have at least a tiny documentation about it.

Yes, I plan to be getting to it. It's a polyalgorithm, which was what sciml_train did before, which mixes different optimizers in a heuristic way to get fast convergence with robustness to local minima.

I really like the SciML ecosystem but I find it in some points quite confusing at the moment.

OptimizationOptimJL can be deleted from that. And what we should do is just have the subpackages reexport Optimization.jl. I don't think the using Zygote is necessary either since that would be done in SciMLSensitivity? So it should be:

using DifferentialEquations, OptimizationPolyalgorithms, SciMLSensitivity, Plots

I was talking to @Vaibhavdixit02 about these usability bits with Optimization.jl, there's a few others as well (missing error messages, error messages bigger than they should be, etc.) and I was hoping to section out some of his time so that he could focus on them. There is the big ACoP conference for Pumas coming up, but shortly after that I think there should be some time to really clean up Optimization.jl's sphere. I agree it's rather necessary. About 90% of issues streaming in are Optimization.jl missing some nicety, so it's definitely on the priority list!

DiffEqFlux = "bridge between ODE and ML"

DiffEqFlux -> SciMLSensitivity. It has actually kind of been that way for 2 years, the deprecation was just a long process. DiffEqFlux now has a more defined role as predefined Flux layers for implicit ML, while the adjoint code to bridge ODEs to ML has lived in SciMLSensitivity for 2 years but we were scared to deprecate until the transition was completed.