DiffEq Optimal Control

[azev77]

Hey, is this package intended to solve general optimal control problems? I really wish I could use NLOptControl.jl but it hasn't been updated to Julia 1.0

[ChrisRackauckas]

I was going to do that, but soon I started helping NLOptControl and didn't do much here. But the most recent work is probably with DiffEqFlux.jl which makes it easy to define the optimization setups to solve in a fast way. https://github.com/JuliaDiffEq/DiffEqFlux.jl/issues/174 is a fairly clear example of how to solve a classic nonlinear control problem for example. Work in https://github.com/JuliaDiffEq/DiffEqFlux.jl/issues/173 would be necessary to make it "good".

I think that when that is all built up, we will return back to this effort and build a high level interface that makes it easy to describe and solve a standard nonlinear optimal control problem by interfacing with that library and BoundaryValueDiffEq.jl for you.

[huckl3b3rry87]

Hey Chris, as you know, I finally got around to updating NLOptControl https://github.com/JuliaMPC/NLOptControl.jl/pull/30.

Should be working on Julia 1.5.

But, I think that NLOptControl was removed from the official Julia package database, because I did not update to Julia 1.0 fast enough;/

[ChrisRackauckas]

Yes, @huckl3b3rry87 glad to see you're back! I definitely would point people over to there for now.

I'm still planning to do this package out, and if you'd like to be a part of the process that would be incredibly helpful. There's just a lot of improvements to the ecosystem in 2020 that we can take advantage of, so I think NLOptControl was great for Julia v0.6 but I want to do a ground-up rewrite that can utilize all of the stuff that has come up since then (and a lot of these features we developed specifically to get around what I think were some of the difficulties of NLOptControl.jl).

Basically, what I want to do is merge the different ways of tackling the same optimization problem in different ways all into one unified interface. I want to do direct, indirect, and variational methods all in one line of code. My approach is going to be to have an OptimalControlSystem which would be a lot like the MTK ODESystem (https://mtk.sciml.ai/dev/systems/ODESystem/) except that it would add a spot for the control variables u and a loss function. The three various ways of solving the optimal control problem would then come as transformations of the OptimalControlSystem into other system types.

For the direct methods, we can have a direct shooting method that transforms into an OptimizationProblem in the style of DiffEqFlux where you just have to choose a universal function approximator. For direct discretize-then-optimize, we can trace DifferentialEquations.jl tableaus to have it spit out the code for the discretization (which I just mentioned to someone on the Slack today), and this would give high order discretizations. This would then build the symbolic form and output an OptimizationSystem, and which then allow for symbolic computations to generate the sparse Hessians and all of that jazz for efficient optimization.

GalacticOptim.jl is being built to unify the Julia optimization libraries so that OptimizationProblem works globally and all of the AD comes for free, which is why we are targeting that. GalacticOptim.jl is planned to interface with MOI.

If anyone wants to join in on this effort, let me know. I think I can lay down the OptimalControlSystem definition and a first pass at some direct methods in a week or so, and that would get us well on our way to something really flexible.

[ChrisRackauckas]

And I plan to probably rename this OptimalControl.jl

[huckl3b3rry87]

Chris,

Thank you for the detailed response! It sounds like you have an exciting problem to tackle, I will definitely help it I can:)

It is nice to look at Julia code again:)

On Wed, Aug 12, 2020, 9:51 PM Christopher Rackauckas < notifications@github.com> wrote:

And I plan to probably rename this OptimalControl.jl

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SciML/DiffEqOptimalControl.jl/issues/4#issuecomment-673254190, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6TAWSTE6TM7GSWQB3Q6HDSANWOBANCNFSM4K75NF6Q .chri

[huckl3b3rry87]

@ChrisRackauckas I noticed that while I am passing tests on both Linux and mac, they are failing due to a precompilation issue on Windows. Unfortunately, I do not have access to a Windows machine and some of my colleges are wanting to use NLOptControl on Windows, but don't have any delving experience. Do you mind taking a quick look to see if there is something simple that is going wrong here? If not,I totally understand, I know that everyone is very busy:)

[freemin7]

I would be interested in helping but i am currently very busy with exams

[alastair-marshall]

I'd also be interested in helping out here wherever possible, although I'm pretty new

[ChrisRackauckas]

Alright, I got the basics implemented in https://github.com/SciML/ModelingToolkit.jl/pull/562 . I think most of the work that's needed right now is that we need GalacticOptim to be much better to really do much with the current transformation. I'll create another transformation tomorrow or this weekend that doesn't require constrained optimization, but this should be the standard one.

[ChrisRackauckas]

From implementing it, I think it's best to do this in ModelingToolkit itself, so I'll close this and if you're interested in this then you should watch ModelingToolkit.jl

[azev77]

FYI, @pulsipher's InfiniteOpt.jl can solve optimal control problems very nicely.

[ChrisRackauckas]

Yeah, but that's all discretize-then-optimize. The issue is building a method-independent format so that it can be optimal in many different ways.

[pulsipher]

To clarify, InfiniteOpt.jl does provide a general interface for modeling infinite-dimensional optimization problems (e.g., optimal control problems) that is independent of the transformation approach used to solve it. We currently only provide discretization methods, but we do feature an API for adding on additional methods that need not be discretization based as described here. Admittedly, the decoupling between the infinite model and the solution approach isn't perfectly decoupled, but we are working on fixing that (https://github.com/pulsipher/InfiniteOpt.jl/issues/105).

Of course the key distinguishing difference is that InfiniteOpt.jl is intended as an interface for general infinite-dimensional optimization problems (i.e., problems with decision functions) and is not a tailored tool for optimal control. Moreover, it is built upon JuMP.jl's API and thus follows a different modeling paradigm than that of ModelingTookit.jl.

I look forward to seeing what comes of this project, it looks pretty cool!

[freemin7]

@pulsipher Just a short question on what you mean with "additional methods that need not be discretization based".

Do you mean "methods that do not discretize in state \cross time space" when saying "additional methods that need not be discretization based"?

For example a vector basis based method like composition of sinosoids such that the system (Fourier basis) is not discretized in state or time but discretized (due to finite RAM and life time of the universe) in frequency space in all practical applicatons.

[pulsipher]

@freemin7 To be more precise InfiniteOpt.jl supports the addition of arbitrary methods to solve infinite-dimensional optimization problems. It provides a modeling abstraction to express general infinite-dimensional models (stored in InfiniteModels) that are independent of the solution/transformation method used to solve them.

Currently, we only natively provide direct transcription techniques to create a TranscriptionModel that can be solved using conventional solvers as provided by JuMP.jl. This discretizes directly over the problem's domain (e.g., state /cross time space). Hence, this is what I was referring to as "discretization based" methods.

So yes, one could extend InfiniteOpt.jl to employ a vector basis method that is discretized in frequency space.

In principle, we could even extend it to employ a general method of weighted residuals technique to transform and solve problems (though that would be a bit of work to implement for arbitrary infinite-dimensional optimization problems which is why we haven't done that yet). However, a particular transformation method could be restricted to a certain sub-group of infinite-dimensional problem classes (e.g., optimal control).

I hope that helps.

[jbcaillau]

Dear all,

I just came across this discussion. Fully agree that it is better to gather efforts than reinventing stuff, although there is of course some trade off between what one wants and what one gets on the shelf.

Given the tremendous amount of ongoing developments in Julia, are you aware of anything else than the previously mentioned packages?

• NLOptControl.jl
• InfiniteOpt.jl

Since AD is now easily available in Julia, we are thinking to move some dev to Julia; mostly things based on indirect / geometric methods (shooting). A glimpse here: https://ct.gitlabpages.inria.fr/gallery

[ChrisRackauckas]

DiffEqFlux is probably the most developed in that sense, with some tutorials describing cases like PDE-constrained optimization with high order adaptive implicit methods. That's the numerical side, and then on the interface side there's some pretty clear advantages to the ModelingToolkit.jl interface so that's the only one it really makes sense to consider there. That said, the MTK control interface isn't quite complete, mostly because of the interaction of nonlinear tearing with symbolic arrays which are being sorted out. We're probably about 6-9 months away from having something that auto-generates sparse parallel adjoints in both direct and indirect methods from one source.

[azev77]

@jbcaillau thanks for posting!

• NLOptControl.jl: had some nice features but isn't really maintained
• InfiniteOpt.jl: an extension of JuMP.jl, is great, @pulsipher did a phenomenal job here. I used it for a simple econ problem.
• As Chris mentioned, it looks like the SciML org is working on optimal control too.
• @jbcaillau Your gallery is impressive. It would be amazing to have you contribute to the Julia ecosystem!

The world would be a much better place if engineers, economists, mathematicians, physicists & everyone else collaborated. For example, the following table summarizes the 4 main control problems economists solve, and they take a similar generic form:

One issue may (or may not) be whether you wanna work on a package that extends an optimization DSL like InfiniteOpt.jl does w/ JuMP.jl, or diffeq solvers etc like SciML's DifferentialEquations.jl.

[ChrisRackauckas]

One issue may (or may not) be whether you wanna work on a package that extends an optimization DSL like InfiniteOpt.jl does w/ JuMP.jl, or diffeq solvers etc like SciML's DifferentialEquations.jl.

I think you have to do both and separate them. The numerical pieces are already put together:

• DiffEqFlux for direct optimize-then-discretize methods and direct AD OtD methods (for example https://diffeqflux.sciml.ai/dev/examples/optimal_control/)
• GalacticOptim for direct discretize-then-optimize methods. It's a bit more general than MOI, but right now you have to discretize by hand 😅.

On the DSL side, there are quite a few reasons to not use JuMP for this:

• JuMP's nonlinear programming modeling tools are... simple at best. JuMP is really good for special forms, a lot less good for nonlinear.
• JuMP doesn't generate good sparse code, and doesn't automatically parallelize the code (MPI, GPU, etc.). https://github.com/mlubin/ReverseDiffSparse.jl was even scrapped awhile back. And it doesn't include structural simplification and nonlinear tearing algorithms.
• JuMP doesn't give an interactive modeling tool for generating better models. It's no SymPy: you have to do a lot of work if you want to write a DSL and make it do good things. For example, what is the optimal way to prove if a user's system is purely polynomial in the JuMP form? In Symbolics.jl it's a one line query because it's made to be a full symbolic system, and so the dev time is orders of magnitude less.
• JuMP symbolics do not trace Julia code. I.e., you cannot automatically transform Julia code to JuMP code and require a DSL. A Symbolics.jl-based system can just automatically transform numerical code for DifferentialEquations.jl to symbolic code, and this can lead to some pretty cool examples and make adoption downstream much easier. Also, it helps in the creation of the library: the discretize-then-optimize code uses the ODE time steppers of DifferentialEquations.jl for the codegen! https://github.com/SciML/ModelingToolkit.jl/blob/v5.20.0/src/systems/control/controlsystem.jl#L145-L191
• Integrating neural networks and stochastic optimization into JuMP is... a task.

So I would say that InfiniteOpt.jl is probably the best DSL you could write on JuMP, but the fundamental limitations of JuMP are the impetus to putting together a wholly new system here built on Symbolics.jl and ModelingToolkit.jl. It's been a fairly long process though, but we're starting to get some results in control finally. One of the last pieces that are making this come together is by not splitting ControlSystem from ODESystem and SDESystem, but instead making there be control variables in every system type. So ControlSystem will go away so that the tearing and all of that apply better. Some of the DtO tooling then needs to be retargeted, but that's not all that hard.