General deep learning solver of agent optimization problem

[sbenthall]

A nice-to-have feature based on the conversation today ...

A general solver to models defined using HARK.frame that uses a (deep) neural network for its solution/policy functions.

The idea is to use Tensorflow to simulation the model forward and use the resulting utility as a loss function, which is then reduced through deep learning training techniques.

Some work by Pablo for inspiration:

https://notes.quantecon.org/submission/5ddb3c926bad3800109084bf

[alanlujan91]

See also

• https://github.com/sischei/DeepEquilibriumNets
• https://github.com/DeepSurrogate/OptionPricing

[sbenthall]

See #1186 -- work on 'frames' has shifted over to 'stages', which have a better design more focused on Bellman equations.

[sbenthall]

https://github.com/google-research/tensorflow_constrained_optimization/blob/master/README.md

[sbenthall]

This can be in HARK.algos and have a similar form after this PR is merged:

https://github.com/econ-ark/HARK/pull/1283

I did a preliminary look into this. I think the key issue might be that the post-value function $v_y$ and the resulting value function $v$ both need to be auto-differentiable for this to be chained from period to period (or stage to stage, or whatever). I believe it's possible for the action-value function (e.g. $u(c) + \beta v_y(m-c)$) to be used as a loss function directly.

[sbenthall]

@alanlujan91 Aha yes the Deep Equilbrium Nets approach seems quite promising :)

[sbenthall]

https://github.com/dseconf/DSE2023/tree/main/06_Scheidegger