Hi, Thank you for the great work on this project. I have a couple of questions regarding the multi-GPU/multi-node training implementation, specifically in the context of the central network.
From my understanding of the source code, it appears that the initial parameters of the actor_critic model on GPU rank_0 are broadcast to other GPU replicas to ensure they hold the same initial parameters. My questions are as follows:
Is broadcasting the initial parameters of the actor_critic model sufficient to ensure that all GPU replicas maintain the same parameters throughout training?
Given that different seeds might be used for each GPU, the central_network could potentially be initialized with different parameter weights. Could this be a potential issue for multi-GPU/multi-node training when using the central_network?
I appreciate your time and assistance in addressing these questions
Hi, Thank you for the great work on this project. I have a couple of questions regarding the multi-GPU/multi-node training implementation, specifically in the context of the central network.
From my understanding of the source code, it appears that the initial parameters of the actor_critic model on GPU rank_0 are broadcast to other GPU replicas to ensure they hold the same initial parameters. My questions are as follows:
Is broadcasting the initial parameters of the actor_critic model sufficient to ensure that all GPU replicas maintain the same parameters throughout training? Given that different seeds might be used for each GPU, the central_network could potentially be initialized with different parameter weights. Could this be a potential issue for multi-GPU/multi-node training when using the central_network? I appreciate your time and assistance in addressing these questions