search

pfnet / pfrl

PFRL: a PyTorch-based deep reinforcement learning library
MIT License
1.18k stars 158 forks source link

About training time #72

Closed zmonoid closed 4 years ago

zmonoid commented 4 years ago

May I know the training speed for a single trial?

In your reproduction section, e.g. DQN:

Training time (in days) across all runs (# domains x # seeds)
--
Mean 3.613

Does this mean that, the training for a single trial is:

200,000,000 frames / (3.613 24 3600 sec) = 640 frames/sec

Above calculation is inline with Deepmind. With default frame skip = 4, the actual speed is:

50,000,000 frames (collected) / (3.613 24 3600 sec) = 160 frames/sec

Is my understanding correct?

Also, may I know your hardware?

prabhatnagarajan commented 4 years ago

Is my understanding correct?

Yes, your understanding is correct. Essentially, the reported time is the average time for running the train_dqn.py script from start to finish (roughly speaking) on a single GPU. There's 200M emulator frames, but 50M timesteps/actions from the agent. So essentially 160 actions per sec (excluding periodic evaluations).

I'll try and inquire more about the compute requirements used for these results.

ummavi commented 4 years ago

As for the hardware, we used a single NVIDIA V100 with 1 Intel Xeon CPU clocked @ 3GHz and 12GB of RAM.

zmonoid commented 4 years ago

@ummavi @prabhatnagarajan Really thanks for your explaination.