search

jurgisp / pydreamer

PyTorch implementation of DreamerV2 model-based RL algorithm
MIT License
205 stars 47 forks source link

Does dreamer need env fixed initialization? #16

Closed CarlDegio closed 1 year ago

CarlDegio commented 1 year ago

I'm doing some robot manipulation tasks by dreamer. But the result now can't get very well (worse than SAC, and don't converge). I suspect it's because my env.reset() lead to random states. At the beginning of each round, the RNN assumes that the previous state is all 0, which seems unreasonable

I found that the paper of PlaNet mentioned that the initial state of the round is fixed. And the dreamer use its RSSM. So I wonder if dreamer can handle randomized starting environments in use.

jurgisp commented 1 year ago

Dreamer can definitely handle randomly initialised environments, the inial zero state is not a problem.

In general if it's not learning well, tweaking entropy coefficient is the first thing to try, and second - kl scale.