[rllib] How to load pre-trained PPO model to evaluate on slightly different environment

[alperyyildiz]

Hello,

I have a trading environment that and PPO agent with custom tf/torch model (I can use both). I use a trainable object which has a step method first train on training set and then validate the trained model on validation set.

To validate the trained model, I need to change timesteps and the dataset used. I tried to achieve this by:

-restoring saved model and changing the environment parameters to allow environment to change dataset and timesteps but ended up with the same training environment -writing to a text file that changed between train/validate on trainable step to send necessary info to the environment to update dataset and timesteps, again ended up with same environment.

Is there any way to achieve this? I read about export_model method that it has some troubles to maintain learnt behaviour. Stucked here for 3 days and any help would be great.

[stale[bot]]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.