search

ikostrikov / pytorch-a2c-ppo-acktr-gail

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).
MIT License
3.56k stars 831 forks source link

FPS calculation #238

Open Xemnas0 opened 4 years ago

Xemnas0 commented 4 years ago

While training, the number of frames used so far is computer as total_num_steps = (j + 1) * args.num_processes * args.num_steps

Shouldn't this be multiplied by the number of stacked frames (default 4)? Also, the number of updates would be num_updates = int( args.num_env_steps) // args.num_steps // args.num_processes // frame_stack

ikostrikov commented 4 years ago

For reporting result I'm using Monitor wrapper from OpenAI: https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail/blob/master/a2c_ppo_acktr/envs.py#L51

It properly computes the number of original environment frames.

Xemnas0 commented 4 years ago

True, but I believe this is not taken into account in the print info while training, is it? FPS therefore should stand for number of environment steps per second, and not frames per second.

ikostrikov commented 4 years ago

Yes, that's true. After the NeurIPS deadline I will change it to "env. steps per second" to avoid any potential confusion.