How to create DPC problem with BlockSSM model

[tudoroancea]

Hi there,

I was looking at the examples in the code and wanted to find something similar to what was done in this paper where you identified the system using a Neural Block SSM and then used this model in the DPC problem. However, none of the provided examples deal with such a configuration and only with Linear SSMs.

In particular, I wanted to know how to load the state_dict for the estimator and the dynamics model.

Thanks a lot in advance !

ted

[drgona]

Hi @tudoroancea thank you for your interest in our code!

At the moment we have one open-source example with system ID using neural ODEs and subsequent control policy training with the DPC algorithm. See the code here: https://github.com/pnnl/neuromancer/blob/master/examples/control/two_tank_sysID_DPC_cl_var_ref.py

The performance needs to be tuned. But the code above gives you a working template for closed-loop policy training with a trained NODE model.

I will create a new open-source example using the BlockSSM model for subsequent DPC soon. Ping me when you don't see anything coming within a week :)

[tudoroancea]

Thanks a lot, that's still something!

For a little bit of context, I was planning to implement a complete sysid/control strategy for an autonomous race car. I am not sure whether NODEs or Neural Block SSMs would be more adapted. Would you have any intuition to give me ? Is one more robust than the other, or easier to train ?

[drgona]

I would favor NODEs because, internally, they make smaller timesteps that are being integrated with the ODE solver instead of direct step-ahead predictions from SSMs. But in the pure black box setting, their performance might depend on the specific problem. In some cases, I have seen SSMs being better in avoiding being stuck in local optima. Another reason for going with NODEs in our framework is that you can make them gray boxes by adding structure in the RHS of the ODE instead of just using a neural net.

In your case, the ODE form of the car dynamics is well known and could be used as a very useful prior for system ID. See the youtube tutorial on parameter estimation for ODE in Neuromancer. https://www.youtube.com/watch?v=HLuqneSnoC8&ab_channel=NeuroMANCER associated code example: https://github.com/pnnl/neuromancer/blob/master/examples/system_identification/duffing_parameter.py

Furthermore, you can combine structure with black box components in a systematic way. See some of our gray box ODE examples in our ode.py library. For instance, the BrusselatorHybrid class takes in the block as a black box nn.Module component that can be used in the RHS of the ODE in a structured way. See the class definition here: https://github.com/pnnl/neuromancer/blob/master/neuromancer/ode.py#L147 You could easily create a similar class for car dynamics.

This approach is called universal differential equations (UDE), developed by Julia folks. See the paper for the reference: https://arxiv.org/abs/2001.04385

[tudoroancea]

Great, thanks for the input, I will keep it in mind !

[ms140429]

Hi Jan @drgona , thanks a lot for this excellent work, very impressive! Do you have any plans to open the code and dataset of the paper "Differentiable Predictive Control: Constrained Deep Learning Alternative to Explicit Model Predictive Control for Unknown Nonlinear Systems". I found this paper is very useful and would like to follow up with more details. Many thanks!

[drgona]

Hi @ms140429, thank you for the inquiry and interest in our work! In this paper, we conducted laboratory experiments with implementations on embedded HW device which involved some custom built; hence it's not directly suitable for open-source. We are working on the expansion of our current set of control examples that will include the model abstractions used in the paper, namely neural state space models and neural control policies for reference tracking. The closest examples to the method in the paper you mentioned are these ones using linear state space models. https://github.com/pnnl/neuromancer/blob/master/examples/control/double_integrator_DPC_ol_fixed_ref.py https://github.com/pnnl/neuromancer/blob/master/examples/control/vtol_DPC_ol_fixed_ref.py

[aarontuor]

@ms140429 We did publish code and the dataset for this paper. It is legacy code (many improvements have been made to neuromancer in the interim) so not recommended to fork development from but can give some details on the exact implementation used in the paper. Data: https://github.com/pnnl/FlexyAirDeepMPC/tree/master/psl/psl/datasets/Flexy_air Code: https://github.com/pnnl/FlexyAirDeepMPC/tree/master/neuromancer/neuromancer/train_scripts