the return value of step function in atari

[lucasliunju]

Hi, I try to run the code with default setting for aatari (Pong). I find the step function (dopamine/dopamine/discrete_domains/atari_lib.py +467) returns five values but the code just defines four values. May I ask how to modify that?

In addition, I find the step function returns 2 values about "done" and I think this maybe the main reason.

Thank you very much!

[JesseFarebro]

Hi @lucasliunju, Dopamine doesn’t support Gym version 0.26. Breaking changes were introduced in v0.26 to the step and reset function (as you pointed out the return signature changed). You should use Gym 0.25.X along with ale-py 0.7.X.

[lucasliunju]

Thank you very much!

May I ask how to train the DQN agent with JAX. I follow this command and I find that is using tensorflow:

python -um dopamine.discrete_domains.train \ --base_dir /tmp/dopamine_runs \ --gin_files dopamine/agents/dqn/configs/dqn.gin

[psc-g]

to run with jax dqn use this instead:

python -um dopamine.discrete_domains.train \ --base_dir /tmp/dopamine_runs \ --gin_files dopamine/jax/agents/dqn/configs/dqn.gin

On Fri, Sep 16, 2022 at 4:03 AM lucasliunju @.***> wrote:

Thank you very much!

May I ask how to train the DQN agent with JAX. I follow this command and I find that is using tensorflow:

python -um dopamine.discrete_domains.train \ --base_dir /tmp/dopamine_runs \ --gin_files dopamine/agents/dqn/configs/dqn.gin

— Reply to this email directly, view it on GitHub https://github.com/google/dopamine/issues/199#issuecomment-1249048691, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE3CCMLWL4TLGCTQUFQW6G3V6QSVXANCNFSM6AAAAAAQMEB3AY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[houeland]

Thanks! I also had this issue, and it now works after manually running pip install ale-py==0.7.5 gym==0.25.2

Can you update the requirements.txt file to specify the versions that dopamine is currently compatible with?

(I followed the instructions which say pip install -r dopamine/requirements.txt, but that ends up with the newer incompatible versions of ale-py and gym, so it currently doesn't work.)

[psc-g]

fyi: with https://github.com/google/dopamine/commit/a2753dae222c75ae991758d4110a84bc01c3215f

i've fixed the version of gym so people don't run into these issues anymore.

On Tue, Sep 20, 2022 at 3:14 PM Tor Gunnar Houeland < @.***> wrote:

Thanks! I also had this issue, and it now works after manually running pip install ale-py==0.7.5 gym==0.25.2

Can you update the requirements.txt file to specify the versions that dopamine is currently compatible with?

(I followed the instructions which say pip install -r dopamine/requirements.txt, but that ends up with the newer incompatible versions of ale-py and gym, so it currently doesn't work.)

— Reply to this email directly, view it on GitHub https://github.com/google/dopamine/issues/199#issuecomment-1252798088, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE3CCMICVK7XZRIRQASSQ7LV7IEKPANCNFSM6AAAAAAQMEB3AY . You are receiving this because you commented.Message ID: @.***>

[lucasliunju]

Thanks for your update.

I try to run the code of jax-based dqn in Atari (Pong) and I find the return is less than the result in this link: https://google.github.io/dopamine/baselines/atari/plots.html

I would like to ask if is it normal.

[psc-g]

what returns are you getting?

On Thu, Sep 22, 2022 at 12:10 AM lucasliunju @.***> wrote:

Thanks for your update.

I try to run the code of jax-based dqn in Atari (Pong) and I find the return is less than the result in this link: https://google.github.io/dopamine/baselines/atari/plots.html

I would like to ask if is it normal.

— Reply to this email directly, view it on GitHub https://github.com/google/dopamine/issues/199#issuecomment-1254500044, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE3CCMNKHGQSGXUERY6EH5TV7PL3JANCNFSM6AAAAAAQMEB3AY . You are receiving this because you commented.Message ID: @.***>