FilippoAiraldi
commented
2 months ago Hi @prajwalvinod,
The environment that I defined here is compatible with stable-baselines-3, so it should not be difficult to play around with the hyperparameters of DDPG, or switch altogether to SAC.
If you want to include also VSL as an action that the RL agent must take, then you have to expand the METANET network to include new links with VSL, and re-define the dynamics (and the environment that makes use of them) to accept not only the ramp metering action, but also the VSL actions as input from the RL agent. The rest should remain more or less the same.
Best of luck!
Filipppo
Hey @FilippoAiraldi,
My name is Prajwal and I am a Masters student in Systems and Control at TU Delft. I am trying to use your code as a base for my traffic simulations for my thesis. I was wondering how one can implement the coordinated control that you described in the following link:
https://github.com/FilippoAiraldi/sym-metanet/blob/dev/examples/coordinated_control_in_casadi.py
with DDPG and SAC while making sure that both ramp metering and vsl actions are taken.
I have gone through both the repositories and I have understood how exactly you have achieved coordinated control with mpc; where you define both actions using a package. But should I want to achieve this using RL, I am having some trouble since I am not entirely sure how I can hardcode that 2 actions need to be taken in such an instance.
I am adding my code which is a derivative of yours which I feel are relevant and hopefully you can give me some insight.
files.zip
Additionally, is there a way to extract and plot data for the RL agents in a similar fashion to the LSTDQ plots ?
Looking forward to your reply.
Regards
Prajwal