Example code doesn't converge

Omegastick / pytorch-cpp-rl

PyTorch C++ Reinforcement Learning

MIT License

511 stars 85 forks source link

Example code doesn't converge #15

Open Omegastick opened 4 years ago

Omegastick commented 4 years ago

Some time around ae030395f56efca50a51335fe4f3367caf950066 we regressed and the example code in gym_client.cpp doesn't converge any more. Presumably because of some difference in our observation normalization compared to the OpenAI Baselines one.

I'll look in more detail this weekend and confirm if it's that exact commit causing the problem.

Omegastick commented 4 years ago

After fixing #14, it converges, but I feel like it used to converge faster. I'll run some more detailed comparisons when I get time.

FPS has dropped about 20% on my laptop since the first release, but that could be a lot of things.