How to load PPO2 model - Githubissues

IBM / rl-testbed-for-energyplus

Reinforcement Learning Testbed for Power Consumption Optimization using EnergyPlus

MIT License

177 stars 74 forks source link

How to load PPO2 model #88

Open yashviagrawal opened 2 years ago

yashviagrawal commented 2 years ago

Hi, I shifted from TRPO to PPO2, and I know how to train the model and save it and then re-train the saved model in TRPO. But, in PPO2, I don't know the code for it.

@antoine-galataud you had once helped me out with the TRPO model, pls can you help me out again?

(an example of the code for trpo policy)

antoine-galataud commented 2 years ago

Hi @yashviagrawal

The learn method signature of PPO2 in OpenAI baselines is close to the TRPO one, w.r.t. saving and loading models. See https://github.com/openai/baselines/blob/ea25b9e8b234e6ee1bca43083f8f3cf974143998/baselines/ppo2/ppo2.py#L21-L24

So basically the same flow applies:

policy = ppo.learn(...)
policy.save("path")
ppo.learn(..., load_path="path")

yashviagrawal commented 2 years ago

@antoine-galataud Actually, for ppo, I can't use the learn function twice, the code doesn't run, this is why I was stuck.

antoine-galataud commented 2 years ago

@yashviagrawal can you share the piece of code that fails and the exception stack?

antoine-galataud commented 1 year ago

@yashviagrawal please have a look at https://github.com/IBM/rl-testbed-for-energyplus/issues/106#issuecomment-1181411454, this should fix your issue too!