search

google-deepmind / acme

A library of reinforcement learning components and agents
Apache License 2.0
3.52k stars 426 forks source link

Regarding Seed for tf rl agents in acme #255

Open kmukeshreddy opened 2 years ago

kmukeshreddy commented 2 years ago

Hello,

Could you please let me know if there is any seed function in acme to fix the TF RL agent randomness? I observed that results vary for each run by 5-10% from the previous run, Is there any way to control this randomness?

Please let me know if any additional details are required.

kmukeshreddy commented 2 years ago

Could you please let me know if there is any update?

I have tried to set the seed by the below code, but the RL agent (D4PG) results still vary for similar runs.

tf.keras.utils.set_random_seed()
tf.config.experimental.enable_op_determinism()
ethanluoyc commented 2 years ago

I believe it may be very difficult to get full determinism in acme. There are a few sources of non-determinism that I imagine would be hard to avoid.

  1. The actor fetches parameters asynchronously.
  2. There is some prefetching in sampling from the reverb which may be non-deterministic.
  3. Reverb sampling is not deterministic. I believe there's currently no way of making sure that reverb samples deterministically.
kmukeshreddy commented 2 years ago

Thank you! for the update.