search

TianhongDai / hindsight-experience-replay

This is the pytorch implementation of Hindsight Experience Replay (HER) - Experiment on all fetch robotic environments.
MIT License
391 stars 77 forks source link

Could you provide me with some advice? #31

Open binbinyouli12 opened 2 months ago

binbinyouli12 commented 2 months ago

Hello,

First of all, thank you for providing the DDPG+HER code; it has been a great help. However, I have some basic questions as I am just starting to learn about reinforcement learning. After adapting your application to my custom environment, I noticed that during the initial stages of training, the printed actor loss is very small, typically around 0.000-something, and the critic loss is usually about 0.0000-something. I am not sure if this is normal or if there is a problem somewhere?

TianhongDai commented 2 months ago

@binbinyouli12 Thank you for using my code - I think it's normal, the most important metric is the success rate.

binbinyouli12 commented 2 months ago

Hello, I have a few more questions. If I only use DDPG in your code, do I just need to remove the HER replacement in the memory part of the code? Another question is, will using HER lead to issues with reward overestimation and failure to correctly guide the agent to the desired location?