Prototype torch.distributed integration

Denys88 / rl_games

RL implementations

MIT License

848 stars 142 forks source link

Prototype torch.distributed integration #158

Closed vwxyzjn closed 2 years ago

vwxyzjn commented 2 years ago

This PR Prototype torch.distributed integration for multi GPU

vwxyzjn commented 2 years ago

cc @markelsanz14, I prototyped the torch.distributed integration but it's only 6% faster. I still feel I am missing the bottleneck somewhere because the prototype with CleanRL was like 25% faster