Reproducability of experiments

[panagiotoum]

Hi,

I am having trouble reproducing the experiments of the PPO with LSTM policy (like the figure in the documentation). What is the exact script that I need to run? I run the for training "simglucose_with_ins_obs_train_ppo_tensorflow.py" file and then for testing the "simulate_controller.py" file.

Thanks for your help.

[MHamza-Y]

The graph was produced using model trained with this script.

Some related notes for reproducing the experiment :

• With current hyper parameters the reward peaks and then collapses. So early stopping is the key
• Alternatively you could reduce lr and entropy coefficient using exponential decay scheduler (e.g decay of 0.999 each iteration)

Also although this algorithm works much better than something naive like PID controller, but the performance is still sub optimal. One guess explaining this sub optimal performance is the continuous action space. If you want better results, you could try discrete action space (something like [0, 0.001, 0.01, 0.1, 1.0, 10.0]). I have not tried it myself but it could improve the performance and reduce training time significantly. Also I found rays rllib to be much better than stable baselines for reinforcement learning.

Hope this helps and best of luck.