search

wujingda / Human-in-the-loop-Deep-Reinforcement-Learning

(Engineering) Toward human-in-the-loop AI: Enhancing deep reinforcement learning via real-time human guidance for autonomous driving
GNU General Public License v3.0
48 stars 10 forks source link

question about actor loss #4

Open EvergrowHook opened 1 year ago

EvergrowHook commented 1 year ago

Hi, In the paper, actor loss is 1681374070503_4CE60915-EE65-45c9-B8AF-3BBA26A44F66 but the code that calculate actor loss for human intervention steps didn't consider the first term (see https://github.com/wujingda/Human-in-the-loop-Deep-Reinforcement-Learning/blob/main/TD3_based_DRL/TD3HUG.py#L148) Also, the human intervention weight in actor loss in the code has a soft update coefficient which isn't included in the paper, and I don't understand what this coefficient is for (see https://github.com/wujingda/Human-in-the-loop-Deep-Reinforcement-Learning/blob/main/TD3_based_DRL/TD3HUG.py#L144). Are these bugs in the code or some tricks that I misunderstand? Looking forward for your help.