Optimization-based shooting solutions

[ChrisRackauckas]

Using the parameter estimation functionality:

http://docs.juliadiffeq.org/latest/analysis/parameter_estimation.html#Parameter-Estimation-1

we can solve BVPs with more equations than solutions by changing problem_new_parameters

http://docs.juliadiffeq.org/latest/analysis/parameter_estimation.html#The-Problem-Generator-Function-1

to change the initial condition (and maybe some parameters) as part of the optimization, and then

http://docs.juliadiffeq.org/latest/analysis/parameter_estimation.html#Note-About-Loss-Functions-1

defining a loss function which is a sum of squares of the BVP condition residuals.

This would allow one to solve BVPs with more conditions than equations in a way that simply minimizes sum squared error.

[rtyasdf]

Since the links above not available, put here relevant ones:

https://diffeq.sciml.ai/stable/analysis/parameter_estimation/#parameter_estimation

https://diffeq.sciml.ai/stable/analysis/parameter_estimation/#The-Problem-Generator-Function

https://diffeq.sciml.ai/stable/analysis/parameter_estimation/#Note-About-Loss-Functions

[avik-pal]

Had a chat with @Vaibhavdixit02 about this recently. Once Optimization.jl has solvers for NLLS (https://github.com/SciML/Optimization.jl/issues/622), the user can pass in any solver (say IPOPT) and we will be able to do an efficient multiple-shooting.

[ChrisRackauckas]

One of the things we need to think about with NLLS is the ability to define equality and inequality constraints too, and then having the possibility to have the shooting loss be handled through equality constraints.

[Vaibhavdixit02]

That can be handled when creating the OptimizationFunction from NLLS, I had it on my mind already :+1:

[avik-pal]

Yes that is on my list of things to do as well. I know Frank and Flemming have a quantum mechanics problem where having that is vital.

I thing to consider is how do we propagate the information. Currently we have a BVProblem and based on that we construct everything. So there are 2 possibilities:

1. Specialize the Multiple Shooting on a NLLS problem and the BVP effectively calls that routine
2. Allow users to specify constraints into the BVP Formulation and propagate that information inth the NLLS Problem