search

qywu / TorchFly

A PyTorch Deep Learning Kit
MIT License
12 stars 5 forks source link

Feature Request Fix DDP torch.jit.trace cannot be pickled #3

Closed qywu closed 4 years ago

qywu commented 4 years ago

Currently, torch.jit.trace can't be pickled with multiprocessing.spawn. To fix this, it is better to use subprocess.Popen instead.

qywu commented 4 years ago

subprocess.Popen is not compatible with jupyter notebook. A better solution is to use cloudpickle instead of the default pickle to serialize the arguments

qywu commented 4 years ago

cloudpickle also can't pickle torch._C.ScriptFunction