search

toshikwa / fqf-iqn-qrdqn.pytorch

PyTorch implementation of FQF, IQN and QR-DQN.
MIT License
161 stars 24 forks source link

Question on QR-DQN calculate_quantile_huber_loss #16

Closed kentropy closed 3 years ago

kentropy commented 3 years ago

When calculating the quantile huber loss in QR-DQN (here), the whole term torch.abs(taus[..., None] - (td_errors.detach() < 0).float()) * element_wise_huber_loss is divided by self.kappa. I cannot find this equation in the paper. Is there any reason for this implementation?

toshikwa commented 3 years ago

I referred to IQN paper. However, there is no difference because kappa = 1.0. Thanks.