search

Jiwonjeon9603 / MASER

This repository is an implementation of "MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffer" accepted to ICML 2022.
20 stars 7 forks source link

Missing explore loss? #2

Closed SigmaBM closed 1 year ago

SigmaBM commented 1 year ago

explore_loss_subset in maser_q_learner.py, i.e., episodic correction losses in paper.

Jiwonjeon9603 commented 1 year ago

Hi, sorry for the late reply!

I'm sorry, and my code was the old version.

Thanks for letting me know, and I updated explore loss in my code.

If there exists more issues, please let me know :)

SigmaBM commented 1 year ago

Thank you for the update!

Now the exploration loss is calculated using target q value, so the gradient won't propagate to q function. Actually, I ran the new released code and found it showed the same result as the old code. Should target q value here be q value from mac_out?