MishaLaskin / curl

CURL: Contrastive Unsupervised Representation Learning for Sample-Efficient Reinforcement Learning
MIT License
561 stars 88 forks source link

Can we integrate the update_critic function and update_cpc function together? #24

Open Sobbbbbber opened 2 years ago

Sobbbbbber commented 2 years ago

Hi, can we integrate the update_critic function and update_cpc function by adding the critic_loss and cpc_loss together? Meanwhile, we only need two optimizers. Is it feasible?

self.cpc_optimizer = torch.optim.Adam([self.CURL.W], lr=encoder_lr) self.critic_optimizer = torch.optim.Adam(self.critic.parameters(), lr=critic_lr, betas=(critic_beta, 0.999)) loss = critic_loss + cpc_loss loss.backward() self.critic_optimizer.step() self.cpc_optimizer.step()

Sobbbbbber commented 2 years ago

In other words, I want to incorporate update_cpc into the update_critic function. Do you have any suggestions, please?