Closed DanielShalam closed 2 years ago
Hi, Thank you for providing this repository. I found it really helpful!
I noticed some errors while training with multi node, and suggest some possible fixes.
First, in 'base.py' method, the calculation of 'num_training_steps' should be changed to:
Second, in 'solo/args/utils.py', the base lr calculation should be changed to:
Thanks!
Thanks for reporting this. We didn't have access to multi-node machines in the past but we are now trying to catch up with that. Feel free to send a PR if you have the hardware to properly test.
Hi, Thank you for providing this repository. I found it really helpful!
I noticed some errors while training with multi node, and suggest some possible fixes.
First, in 'base.py' method, the calculation of 'num_training_steps' should be changed to:![image](https://user-images.githubusercontent.com/55020509/168784360-062bf94a-9bd8-4859-9b6b-c21ca64df59b.png)
Second, in 'solo/args/utils.py', the base lr calculation should be changed to:![image](https://user-images.githubusercontent.com/55020509/168785103-e7963dff-9ec6-4485-852b-99e5600c75f0.png)
Thanks!