Closed vwxyzjn closed 2 years ago
This PR fixes multi-gpu bug w/ central_value.
central_value
Tested w/ torchrun --standalone --nnodes=1 --nproc_per_node=2 train.py multi_gpu=True task=Ant task=ShadowHandOpenAI_LSTM num_envs=2048 wandb_activate=True wandb_entity=costa-huang train.params.config.minibatch_size=8192
torchrun --standalone --nnodes=1 --nproc_per_node=2 train.py multi_gpu=True task=Ant task=ShadowHandOpenAI_LSTM num_envs=2048 wandb_activate=True wandb_entity=costa-huang train.params.config.minibatch_size=8192
Which yields this run
This PR fixes multi-gpu bug w/
central_value.Tested w/
torchrun --standalone --nnodes=1 --nproc_per_node=2 train.py multi_gpu=True task=Ant task=ShadowHandOpenAI_LSTM num_envs=2048 wandb_activate=True wandb_entity=costa-huang train.params.config.minibatch_size=8192Which yields this run