Performance Issue: Significantly Slower Training Speed with PPO in cartpole_simulink Example Compared to Gym's CartPole-v1

[Mingzefei]

Hello,

I have been working with the simulink_gym project, specifically the cartpole_simulink example, to train a reinforcement learning agent using the PPO strategy. However, I observed that the training process is approximately 20 times slower compared to the standard Gym implementation of CartPole-v1. I don't know if this is normal.

Is this level of performance discrepancy expected due to the nature of the integration between Python and Simulink, or is there a potential optimization issue that could be addressed? Additionally, are there any recommended approaches or modifications to enhance the performance of Python+Simulink setups for reinforcement learning applications?

Any insights or suggestions you could provide would be greatly appreciated as I continue to explore this integration for control systems training.

Thank you!

[johbrust]

Hi (and sorry for my late response),

performance (e.g., training steps per second) was not on my agenda for now. I somehow expected the Simulink Gym implementation to be slower due to the bottleneck of the TCP communication between the software packages (i.e., Python and Matlab/Simulink). There probably is room for improvement.

I will keep this issue open as a reminder to look into possible performance improvements.