search

MishaLaskin / curl

CURL: Contrastive Unsupervised Representation Learning for Sample-Efficient Reinforcement Learning
MIT License
561 stars 88 forks source link

Momentum update is never used in the code #26

Open datonefaridze opened 1 year ago

datonefaridze commented 1 year ago

Hi, I was going thought the code and couldn't find where momentum encoder was being updated, I think it is initialized only once at the beginning and then isn't trained at all

Chengyui commented 1 year ago

Cause the CURL encoder is a part of the critic and critic_target. In SAC, critic and critic_target need to be soft_update. So there is no need to do it again in CURL.