DLR-RM / stable-baselines3

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.
https://stable-baselines3.readthedocs.io
MIT License
9.13k stars 1.7k forks source link

How can i change a Distribution? #2032

Open CAI23sbP opened 1 week ago

CAI23sbP commented 1 week ago

❓ Question

How are you @araffin ? I have two questions

  1. If I want to change distribution , how can I do it?

Here is my situation I made an agent that was based on PPO(diagonal dist) by imitation learning library( behavioral cloning) And then, Finetuning the pretrained model use a 'gSDE' (StatedependentDist) It is right to change a sde_sample_freq from 1 to n?

  1. How can i create a normalized observation(for demonstration data)? Is it okay to use a VecNormEnv? Might i get any hit about the above issues?

Thank you!

Checklist

araffin commented 1 week ago

It is right to change a sde_sample_freq from 1 to n?

should be fine

Is it okay to use a VecNormEnv?

that's one solution. But if you have a fixed dataset, you can precompute the normalization.