galnov
commented
5 years ago The -n flag enables single-node multi-thread training, where multiple agents are training in parallel. It brings speed advantage vs. the single-node single-thread training, as demonstrated in Figure 4 of the Asynchronous Methods for Deep Reinforcement Learning paper.
Can I understand -n command as an extension to multi-seed running? It seems to me that multi-processing is like creating more uncorrelated samples for training, and run multiple tests (possibly with different seeds, I haven't looked into details). I know that in many papers, different-seed results are often required. Since the timestep logging for each worker are same, I assume -n doesn't bring speed advantage but lower variance and possible advantages based on this, right? Thank you!