Hi,
I have a question about how to set seed when using multi gpus.
The following code snippet uses torch.cuda.manul_seed funciton.
if is_cuda and seed > 0:
# These ensure same initialization in multi gpu mode
torch.cuda.manual_seed(seed)
But I found a warning in the document of torch.cuda.manual_seed, which says "If you are working with a multi-GPU model, this function is insufficient to get determinism. To seed all GPUs, use manual_seed_all()."
Would it be better to change torch.cuda.manual_seed(seed) to torch.cuda.manual_seed_all(seed) ?
Hi, I have a question about how to set seed when using multi gpus.
The following code snippet uses
torch.cuda.manul_seed
funciton.But I found a warning in the document of torch.cuda.manual_seed, which says "If you are working with a multi-GPU model, this function is insufficient to get determinism. To seed all GPUs, use manual_seed_all()."
Would it be better to change
torch.cuda.manual_seed(seed)
totorch.cuda.manual_seed_all(seed)
?