Open liyifan20010515 opened 6 months ago
出现了多进程can‘t pickle问题 train_loader = DataLoader(train_dataset, sampler=LadderSampler(train_dataset, batch_size), num_workers=num_workers, batch_size=batch_size, collate_fn=lambda e: TPGDataset.collect_fn_quadkey(e, train_dataset, sampler, self.QUADKEY, self.loc2quadkey, k=num_neg, N_nearest=self.N_nearest, mix=mix, train=True, length=self.length)) 报错信息如下： EOFError: Ran out of input return _MultiProcessingDataLoaderIter(self) File "C:\some files\TPG-main\Bigscity-LibCity\venv\lib\site-packages\torch\utils\data\dataloader.py", line 914, in init w.start() File "C:\ruanjan\python 3.7\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self) File "C:\ruanjan\python 3.7\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\ruanjan\python 3.7\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\ruanjan\python 3.7\lib\multiprocessing\popen_spawn_win32.py", line 89, in init reduction.dump(process_obj, to_child) File "C:\ruanjan\python 3.7\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'TPGDataset.get_data..' 请问应该怎么修改呢，是不是不能这样使用
出现了多进程can‘t pickle问题 train_loader = DataLoader(train_dataset, sampler=LadderSampler(train_dataset, batch_size), num_workers=num_workers, batch_size=batch_size, collate_fn=lambda e: TPGDataset.collect_fn_quadkey(e, train_dataset, sampler, self.QUADKEY, self.loc2quadkey, k=num_neg, N_nearest=self.N_nearest, mix=mix, train=True, length=self.length)) 报错信息如下： EOFError: Ran out of input return _MultiProcessingDataLoaderIter(self) File "C:\some files\TPG-main\Bigscity-LibCity\venv\lib\site-packages\torch\utils\data\dataloader.py", line 914, in init w.start() File "C:\ruanjan\python 3.7\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self) File "C:\ruanjan\python 3.7\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\ruanjan\python 3.7\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\ruanjan\python 3.7\lib\multiprocessing\popen_spawn_win32.py", line 89, in init reduction.dump(process_obj, to_child) File "C:\ruanjan\python 3.7\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'TPGDataset.get_data..'
请问应该怎么修改呢，是不是不能这样使用