search

google-deepmind / reverb

Reverb is an efficient and easy-to-use data storage and transport system designed for machine learning research
Apache License 2.0
700 stars 93 forks source link

Reverb error while running DQN #43

Closed Idate96 closed 3 years ago

Idate96 commented 3 years ago

I am training a DQN agent in acme but after a while in the training I get the following error (or info??) that gets repeated over and over 

[reverb/cc/selectors/prioritized.cc:213] Tree needs to be initialized because node with index 0 has approximation error 0.000137329, which exceeds the threshold of 0.0001

I searched in the code but could not find much. Any idea what's the cause of this? What's the approximation error of a node?

acassirer commented 3 years ago

Hey,

The message looks a bit more scary than it is and should be treated as info (i.e. not error). This message is coming from the PrioritizedSelector and is emitted when enough floating point error has been aggregated to motivate a rebalancing of the sum tree. This all happens automatically and generally don't require any action on your side.

That being said, this message should not appear very frequently. The message signals that an expensive operation (O(NlogN) rather than O(logN)) so if it appears very frequently and your experiment is bottlenecked by Reverb then it might be worth while rethinking the magnitude of the priorities you are using. I'd say that this is very unlikely to be a problem but wanted to mention it just in case.