Using current seed fixing code, it may cause seed doesn't fix when we use multi-gpu.
Why
Current seed fixing code uses torch.cuda.manual_seed(seed) = True. However, from PyTorch official docs, manual_seed function gets only current_device not counts devices and fix seeds all of devices. Instead, manual_seed_all function counts devices and fix seeds all of devices. You can reference PyTorch manual_seed function codes and PyTorch manual_seed_all function codes.
How
[x] Change torch.cuda.manual_seed(seed) = True to torch.cuda.manual_seed_all(seed) = True
What
Why
Current seed fixing code uses
torch.cuda.manual_seed(seed) = True. However, from PyTorch official docs, manual_seed function gets onlycurrent_devicenot counts devices and fix seeds all of devices. Instead, manual_seed_all function counts devices and fix seeds all of devices. You can reference PyTorch manual_seed function codes and PyTorch manual_seed_all function codes.How
torch.cuda.manual_seed(seed) = Truetotorch.cuda.manual_seed_all(seed) = True