search

google-research / seed_rl

SEED RL: Scalable and Efficient Deep-RL with Accelerated Central Inference. Implements IMPALA and R2D2 algorithms in TF2 with SEED's architecture.
Apache License 2.0
798 stars 146 forks source link

Is the gcp/train_atari.sh script actually using one GPU device for training? #62

Closed bingykang closed 3 years ago

bingykang commented 3 years ago

According to the script https://github.com/google-research/seed_rl/blob/master/gcp/train_atari.sh#L32-L39, it requires two NVIDIA_TESLA_P100 gpu.

But in this script https://github.com/google-research/seed_rl/blob/master/common/utils.py#L99, when there are not TPU devices, it will adopt OneDeviceStrategy for learning.

So, this gcp/train_atari.sh script is actually the same as running seed rl in a local manner?

lespeholt commented 3 years ago

I believe you're right about that. E.g. see the comment just before:

n1-highmem-32 provides 208GBs of RAM, n1-highmem-16 provides 104GBS.

Training on ATARI requires a bit more than 104GBs due to the large replay

buffer, so we need n1-highmem-32, which requires 2 GPUs (see # https://cloud.google.com/ml-engine/docs/using-gpus).

One could arguably use the two GPUs efficiently instead of wasting one.