Prototype torch.distributed

Denys88 / rl_games

RL implementations

MIT License

848 stars 142 forks source link

Prototype torch.distributed #165

Closed vwxyzjn closed 2 years ago

vwxyzjn commented 2 years ago

Follow up on #158... Weird git issues.

This is a draft.

vwxyzjn commented 2 years ago

Looking further into the bottlenect:

The left is CleanRL, right is rl-games. It seems the all_reduce cost is really high: maybe it's related to NN architecture? Will further look into it.

Denys88 commented 2 years ago

oh, thats a huge difference. what architecture do you have?

vwxyzjn commented 2 years ago

Could be the architecture: I use the default one for Ant. After controlling the NN settings to be the same, they look more similar.