search

google / dopamine

Dopamine is a research framework for fast prototyping of reinforcement learning algorithms.
https://github.com/google/dopamine
Apache License 2.0
10.42k stars 1.36k forks source link

Jax vs normal agent #153

Open satst27 opened 3 years ago

satst27 commented 3 years ago

Hi,

I think it will helpful to clarify that the checkpoints that you have provided are for the Jax verion and not the normal version:

"The TensorFlow checkpoint files for 5 independent runs of the 4 agents on all 60 games are available. The format for each of the files is: https://storage.cloud.google.com/download-dopamine-rl/lucid/${AGENT}/${GAME}/${RUN}/tf_ckpt-199.${SUFFIX}, where:

AGENT can be "dqn", "c51", "rainbow", or "iqn"."

Are there checkpoints available for all games for non-Jax version as well?

thanks, satst27

psc-g commented 3 years ago

hi, the checkpoints are for the tensorflow version, not for the jax version. we don't have checkpoints for all games and all agents on JAX, but i'm hoping to be able to release the quantile checkpoints on JAX sometime in the near future (since quantile is only available on JAX).

On Wed, Oct 7, 2020 at 5:20 AM satst27 notifications@github.com wrote:

Hi,

I think it will helpful to clarify that the checkpoints that you have provided are for the Jax verion and not the normal version:

"The TensorFlow checkpoint files for 5 independent runs of the 4 agents on all 60 games are available. The format for each of the files is: https://storage.cloud.google.com/download-dopamine-rl/lucid/${AGENT}/${GAME}/${RUN}/tf_ckpt-199.${SUFFIX}, where:

AGENT can be "dqn", "c51", "rainbow", or "iqn"."

Are there checkpoints available for all games for non-Jax version as well?

thanks, satst27

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/dopamine/issues/153, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE3CCMM2473JHHVYL6LTOZDSJQXHDANCNFSM4SHED4DQ .

houeland commented 1 year ago

Bump, is this actually completed now? I see that e.g. https://console.cloud.google.com/storage/browser/_details/download-dopamine-rl/jax/quantile/Pong/1/ckpt.199 exists now.

(I currently only see 6 games there, but at least it's not 0. download-dopamine-rl/jax/dqn_adam_mse looks like it actually has checkpoints for all the trained games(?), while download-dopamine-rl/jax/iqn has 0.)

psc-g commented 1 year ago

sorry for delay, will get to this after the iclr deadline! the dqn_adam_mse folder does have for all games, but as the directory suggests it is with Adam optimizer and MSE loss (as we explored in Revisiting Rainbow: http://proceedings.mlr.press/v139/ceron21a.html)

we haven't released checkpoints externally for all games for the jax agents. after the iclr deadline i'll see about releasing all of them.

On Tue, Sep 20, 2022 at 3:12 PM Tor Gunnar Houeland < @.***> wrote:

Bump, is this actually completed? I see that e.g. https://console.cloud.google.com/storage/browser/_details/download-dopamine-rl/jax/quantile/Pong/1/ckpt.199 exists now.

(I currently only see 6 games there, but at least it's not 0. download-dopamine-rl/jax/dqn_adam_mse looks like it actually has checkpoints for all the trained games(?), while download-dopamine-rl/jax/iqn has 0.)

— Reply to this email directly, view it on GitHub https://github.com/google/dopamine/issues/153#issuecomment-1252796273, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE3CCMIUNQ5JNYQ7ZHKQCPDV7IEDBANCNFSM4SHED4DQ . You are receiving this because you commented.Message ID: @.***>