search

SciML / ModelingToolkit.jl

An acausal modeling framework for automatically parallelized scientific machine learning (SciML) in Julia. A computer algebra system for integrated symbolics for physics-informed machine learning and automated transformations of differential equations
https://mtk.sciml.ai/dev/
Other
1.43k stars 207 forks source link

Formulating and solving intertemporal optimization problems #372

Closed finmod closed 3 years ago

finmod commented 4 years ago

Formulate intertemporal optimization problems in continuous time under both certainty (deterministic DEs) and uncertainty (stochastic DEs).

The scope is to formulate multvariate systems of ODEs and PDEs at the elementary level of an agent, a sector, a country based on a generalized Bolza problem. The elements can then be scaled up in terms of many state and control variables and aggregated to a meta level by composition of elements (100+ as in NeuralNetDiffEq).

The strength of ModelingToolkit is to apply vector differential operator algebra to calculus of variation, maximum principle, dynamic programming, mean field games problems to output ODE and PDE problems that can be processed elsewhere in the DiffEq ecosystem: solvers, sensitivity analysis etc. The available operators should be: Lagrangian, inner product brackets, Hamiltonian, Euler-Lagrange, gradient and divergence, Laplacian, Poisson bracekets, Legendre transform, transversality conditions or limits. The objective is to have a coherent mathematical notation throughout DiffEq as in "Calculus of Variation and Optimal Control Theory" of Daniel Liberzon or other agreeable and consistent notation. A good vision of presentation is QuantumOptics.jl except that we do not deal in quantum physics here.

The most general intertemporal optimization template setup is the generalized Bolza problem:

Generalized Bolza

from http://staff.matapp.unimib.it/~calogero/pdf/note_optimal_control.pdf. The problem is then categorized as fixed end points, variable end-points, free final state, infinite-horizon etc. One should be able to specify analytical functional forms depending on parameters for f(t,x,u) like CARA, CRRA utilities, production functions, cost and profit functions. Transition equations g(t,x,u) can be deterministic, with measurement noise, or various form of stochastic. Finally split out in mathematical terms the transition dynamics to pass on the appropriate solver to generate a set of trajectories.

Other relevant references: Applied Intertemporal Optimization of Walde: https://www.waelde.com/pdf/AIO.pdf HJB Approach to some Optimal Control Problems : https://pastel.archives-ouvertes.fr/tel-01275676/document Miranda and Fackler: CompEcon toolbox in Matlab for Continuous time deterministic optimal control https://pfackler.wordpress.ncsu.edu/compecon/154-2/

@ChrisRackauckas where to start or continue from with ModelingToolkit?

ChrisRackauckas commented 4 years ago

I think the place to start would be creating a system type which handles the representation of the problem, and then having the most basic transformation to a solvable form. After that, there's a million things that can be done of course which would all be transformations from the OptimalControlSystem type.

Uroc327 commented 4 years ago

@finmod do you already have a branch where you have worked on this? I'd be interested as well.

finmod commented 4 years ago

No, I don't because I do not have a deep enough knowledge and documentation of the building components of ModelingTookit to proceed to defining an OptimalControlSystem type.

ChrisRackauckas commented 3 years ago

A ControlSystem type exists, and I think we'll try refining this down to simpler issues on that type.