critic update frequency

[a-z-e-r-i-l-a]

in DRQAgent.update it seems that the critic is updated at every environment step which makes sampling from the environment rather slow.

Do you think if it's safe adding an additional frequency to the critic update?

[denisyarats]

Yes, you can try that, we haven't experimented with this.