Closed jinserk closed 4 years ago
@jinserk
This bug seems like not related to matorage and seems to be related to pytorch.
Can you show me how to initialize torch's DataLoader
? Did you use DistributedSampler
properly?
Hmm.. In the above case, I just produced one forked process so DistributedSampler
was not set.. Will check it more if the error is not related to matorage
. Sorry and thanks for letting me know.
@jinserk Thank you. If it becomes a problem with matorage, we will re-open it again. :)
Also, I would appreciate it if you always leave an issue with even a minor one.
Another bug(?) report here. I'm using PyTorch DDP so used the
matorage.torch.Dataset
in a forked process (multiprocessing forkserver is set). Of course the dataset was initialized inrun()
function to avoid unexpected pickle error. However, I still get an error related to the picklization:I have no idea what the
WeakValueDictionary
means. Did you test the use ofDataset
and the resultantDataLoader
in a multiprocessing environment?Thank you again!