AttributeError: Can't pickel local object 'get_dataset.<locals>.Dataset'

[Innan666]

(measure) C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src>python main.py ctdet --exp_id 640 --batch_size 4 --dataset corner --input_res 640 --num_epochs 150 --val_intervals 10 Fix size testing. training chunk_sizes: [4] The output will be saved to C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib....\exp\ctdet\640 heads {'hm': 80, 'wh': 2, 'reg': 2} Namespace(K=100, aggr_weight=0.0, agnostic_ex=False, arch='dla_34', aug_ddd=0.5, aug_rot=0, batch_size=4, cat_spec_wh=False, center_thresh=0.1, chunk_sizes=[4], data_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..\data', dataset='corner', debug=0, debug_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..\exp\ctdet\640\debug', debugger_theme='white', demo='', dense_hp=False, dense_wh=False, dep_weight=1, dim_weight=1, down_ratio=4, eval_oracle_dep=False, eval_oracle_hm=False, eval_oracle_hmhp=False, eval_oracle_hp_offset=False, eval_oracle_kps=False, eval_oracle_offset=False, eval_oracle_wh=False, exp_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..\exp\ctdet', exp_id='640', fix_res=True, flip=0.5, flip_test=False, gpus=[0], gpus_str='0', head_conv=256, heads={'hm': 80, 'wh': 2, 'reg': 2}, hide_data_time=False, hm_hp=True, hm_hp_weight=1, hm_weight=1, hp_weight=1, input_h=640, input_res=640, input_w=640, keep_res=False, kitti_split='3dop', load_model='', lr=0.000125, lr_step=[90, 120], master_batch_size=4, mean=array([[[0.40789655, 0.44719303, 0.47026116]]], dtype=float32), metric='loss', mse_loss=False, nms=False, no_color_aug=False, norm_wh=False, not_cuda_benchmark=False, not_hm_hp=False, not_prefetch_test=False, not_rand_crop=False, not_reg_bbox=False, not_reg_hp_offset=False, not_reg_offset=False, num_classes=80, num_epochs=150, num_iters=-1, num_stacks=1, num_workers=4, off_weight=1, output_h=160, output_res=160, output_w=160, pad=31, peak_thresh=0.2, print_iter=0, rect_mask=False, reg_bbox=True, reg_hp_offset=True, reg_loss='l1', reg_offset=True, resume=False, root_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..', rot_weight=1, rotate=0, save_all=False, save_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..\exp\ctdet\640', scale=0.4, scores_thresh=0.1, seed=317, shift=0.1, std=array([[[0.2886383 , 0.27408165, 0.27809834]]], dtype=float32), task='ctdet', test=False, test_scales=[1.0], trainval=False, val_intervals=10, vis_thresh=0.3, wh_weight=0.1) Der Befehl "cp" ist entweder falsch geschrieben oder konnte nicht gefunden werden. Creating model... Setting up data... ==> initializing coco 2017 val data. loading annotations into memory... Done (t=0.00s) creating index... index created! Loaded val 177 samples ==> initializing coco 2017 train data. loading annotations into memory... Done (t=0.03s) creating index... index created! Loaded train 934 samples Starting training... ctdet/640Traceback (most recent call last): File "main.py", line 102, in main(opt) File "main.py", line 70, in main log_dicttrain, = trainer.train(epoch, train_loader) File "C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\trains\base_trainer.py", line 119, in train return self.run_epoch('train', epoch, data_loader) File "C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\trains\base_trainer.py", line 61, in run_epoch for iter_id, batch in enumerate(data_loader): File "C:\Users\IFL\Anaconda3\envs\measure\lib\site-packages\torch\utils\data\dataloader.py", line 193, in iter return _DataLoaderIter(self) File "C:\Users\IFL\Anaconda3\envs\measure\lib\site-packages\torch\utils\data\dataloader.py", line 469, in init w.start() File "C:\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "C:\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\popen_spawn_win32.py", line 65, in init reduction.dump(process_obj, to_child) File "C:\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'get_dataset..Dataset'

When I train on my own dataset there is an ERROR. Could anyone help me whats wrong?

[xingyizhou]

Thanks for adopting this code for multi-processing. Pytorch multi-processing requires a fixed dataset class, while in our case the dataset class is created dynamically here. A ugly but simple way to do this is to manually and explicitly create the specific combination like:

class Dataset(dataset_factory['coco'], _sample_factory['ctdet']):
  pass

def get_dataset(dataset, task):
  return Dataset

[Innan666]

(measure) C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src>python main.py ctdet --exp_id 640 --batch_size 4 --lr 1.5625e-5 --dataset corner --input_res 640 --num_epochs 150 --val_intervals 10 Fix size testing. training chunk_sizes: [4] The output will be saved to C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib....\exp\ctdet\640 heads {'hm': 1, 'wh': 2, 'reg': 2} Namespace(K=100, aggr_weight=0.0, agnostic_ex=False, arch='dla_34', aug_ddd=0.5, aug_rot=0, batch_size=4, cat_spec_wh=False, center_thresh=0.1, chunk_sizes=[4], data_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..\data', dataset='corner', debug=0, debug_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..\exp\ctdet\640\debug', debugger_theme='white', demo='', dense_hp=False, dense_wh=False, dep_weight=1, dim_weight=1, down_ratio=4, eval_oracle_dep=False, eval_oracle_hm=False, eval_oracle_hmhp=False, eval_oracle_hp_offset=False, eval_oracle_kps=False, eval_oracle_offset=False, eval_oracle_wh=False, exp_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..\exp\ctdet', exp_id='640', fix_res=True, flip=0.5, flip_test=False, gpus=[0], gpus_str='0', head_conv=256, heads={'hm': 1, 'wh': 2, 'reg': 2}, hide_data_time=False, hm_hp=True, hm_hp_weight=1, hm_weight=1, hp_weight=1, input_h=640, input_res=640, input_w=640, keep_res=False, kitti_split='3dop', load_model='', lr=1.5625e-05, lr_step=[90, 120], master_batch_size=4, mean=array([[[0.40789655, 0.44719303, 0.47026116]]], dtype=float32), metric='loss', mse_loss=False, nms=False, no_color_aug=False, norm_wh=False, not_cuda_benchmark=False, not_hm_hp=False, not_prefetch_test=False, not_rand_crop=False, not_reg_bbox=False, not_reg_hp_offset=False, not_reg_offset=False, num_classes=1, num_epochs=150, num_iters=-1, num_stacks=1, num_workers=4, off_weight=1, output_h=160, output_res=160, output_w=160, pad=31, peak_thresh=0.2, print_iter=0, rect_mask=False, reg_bbox=True, reg_hp_offset=True, reg_loss='l1', reg_offset=True, resume=False, root_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..', rot_weight=1, rotate=0, save_all=False, save_dir='C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\..\..\exp\ctdet\640', scale=0.4, scores_thresh=0.1, seed=317, shift=0.1, std=array([[[0.2886383 , 0.27408165, 0.27809834]]], dtype=float32), task='ctdet', test=False, test_scales=[1.0], trainval=False, val_intervals=10, vis_thresh=0.3, wh_weight=0.1) Der Befehl "cp" ist entweder falsch geschrieben oder konnte nicht gefunden werden. Creating model... Setting up data... ==> initializing corner val data. loading annotations into memory... Done (t=0.00s) creating index... index created! Loaded val 177 samples ==> initializing corner train data. loading annotations into memory... Done (t=0.02s) creating index... index created! Loaded train 934 samples Starting training... ctdet/640Traceback (most recent call last): File "main.py", line 102, in main(opt) File "main.py", line 70, in main log_dicttrain, = trainer.train(epoch, train_loader) File "C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\trains\base_trainer.py", line 119, in train return self.run_epoch('train', epoch, data_loader) File "C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\trains\base_trainer.py", line 61, in run_epoch for iter_id, batch in enumerate(data_loader): File "C:\Users\IFL\Anaconda3\envs\measure\lib\site-packages\torch\utils\data\dataloader.py", line 582, in next return self._process_next_batch(batch) File "C:\Users\IFL\Anaconda3\envs\measure\lib\site-packages\torch\utils\data\dataloader.py", line 608, in _process_next_batch raise batch.exc_type(batch.exc_msg) AttributeError: Traceback (most recent call last): File "C:\Users\IFL\Anaconda3\envs\measure\lib\site-packages\torch\utils\data_utils\worker.py", line 99, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "C:\Users\IFL\Anaconda3\envs\measure\lib\site-packages\torch\utils\data_utils\worker.py", line 99, in samples = collate_fn([dataset[i] for i in batch_indices]) File "C:\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\datasets\sample\ctdet.py", line 39, in getitem height, width = img.shape[0], img.shape[1] AttributeError: 'NoneType' object has no attribute 'shape'

Thanks for your help. But i got a new Error. It seems like the Images are not readed. Do you know what's the problem?

[fan945327815]

I have the same problem as you.Have you finally solved it?

[fan945327815]

I know, it's the name of floder

[LigZhong]

I know, it's the name of floder

I met the same problem, can you explain a little bit more on fixing this? thank you very much.

[ZHANG-hengzhi]

(测量)C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src>python main.py ctdet--exp_id 640--批处理_4-数据集角--输入_res 640--num_EIRCHS 150--Val_间隔10 修正尺寸测试。 训练块大小：[4] 输出将保存到C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib....\exp\ctdet\640 头{‘hm’：80，‘wh’：2，‘reg’：2} 命名空间(K=100，aggr_wh=0.0，不可知论_ex=false，ARCH=‘DLA_34’，Aug_DDD=0.5，Aug_ROT=0，批处理_size=4，cat_spec_wh=false，心_阈值=0.1，区块_大小=[4]，data_dir=‘C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib....\data’，数据集=‘角’，调试=0，debug_dir=‘C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib....\exp\ctdet\640\debug’，调试器_SUBATEY=‘White’，DEPE_hp=false，密集_wh=false，dep_WH=1，DUMP_WW=1，DUP_REACH=4，EALE_ORACLE_DEP=false，EALE_ORCLE_hmhp=false，val_ORCLE_hmhp=false，val_oracle_hp_kps=假，val_oracle_wh=false，exp_dir=‘C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib....\exp\ctdet’，exp_id=‘640’，FIX_RES=True，FLIP=0.5，FLOP_test=false，GPU=[0]，GPUS_str=‘0’，头_conv=256，头={‘hm’：80，‘wh’：2，‘reg’：2}，隐藏_数据_时间=假，hm_hp=真，hm_hp_权重=1，输入_h=640，输入_s=640，保持_res=假，Kitti_Split=‘3 dop’，Load_model=0.000125，LR_Step=[90,120]，母批大小=4，均值=数组([[0.40789655，0.44719303，0.47026116]]，Dtype=‘loat 32’，mse_rand_flow=false，nms=false，no_color_aug=false，Norm_wh=false，not_Cuda_Benchmark=false，not_hm_hp=false，not_preFETCE_test=false，NOT_rand_FROW=false，NOT_REG_BBOX=false，NOT_REG_hp_偏移=false，NOREG_偏移=假，Num_class=80，Num_EIRCHS=150，num_iters=-1，num_stack=1，num_工=4，OFF_1，输出_h=160，输出_RES=160，输出=160，打印峰值=0，REG_LOAD=‘L1’，REG_偏移=真，简历=假，root_dir=‘C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib....’，ROUT_WITH=1，旋转=0，保存_ALL=假，save_dir=‘C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib....\exp\ctdet\640’，标度=0.4，分数_阈值=0.1，种子=317，移位=0.1，std=数组([[0.2886383，0.27408165，0.27809834]]，dtype=Float 32)，任务=‘ctdet’，test=false，test_Scale=[1]0]，trainval=false，Val_区间=10，Vis_阈值=0.3，wh_ww=0.1) Befehl“cp”列表进料器Falsch geschrieben oder 不知道该怎么做。 创造模型..。 设置数据..。 ==>初始化coco 2017 Val数据。 将注释加载到内存中..。 已完成(t=0.00s) 创建索引..。 创建索引！ 负载Val 177样品 ==>初始化coco 2017列车数据。 将注释加载到内存中..。 已完成(t=0.03s) 创建索引..。 创建索引！ 加载列车934样本 开始训练..。 Ctdet/640 trraceback(最近一次调用)： 文件“main.py”，第102行，在 Main(OPT) 文件“main.py”，第70行，在main中 火车(时代，火车装载机) 文件“C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\trains\base_trainer.py”，第119行，在火车上 返回.Run_EIRCH(“列车”、“时代”、“数据装载机”) 文件“C：\Users\IFL\Desktop\Liu\Detector\CenterNet\src\lib\trains\base_trainer.py”，第61行，在Run_EIRCH中 对于ITER_id，批处理枚举(Data_Loader)： 文件“C：\Users\IFL\Anaconda3\envs\measure\lib\site-packages\torch\utils\data\dataloader.py”，第193行，在ITER 返回DataLoaderIter(Self) 文件“C：\Users\IFL\Anaconda3\envs\measure\lib\site-packages\torch\utils\data\dataloader.py”，第469行，在依尼特 W.start() 文件“C：\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\process.py”，第105行，开始 自._popen=Self._Popen(Self) 文件“C：\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\context.py”，第223行，in_popen 返回_default_context.get_context().Process._Popen(process_obj) 文件“C：\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\context.py”，第322行，in_popen 返回Popen(Process_Obj) 文件“C：\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\popen_spawn_win32.py”，第65行，在依尼特 还原.转储(Process_obj，to_child) 文件“C：\Users\IFL\Anaconda3\envs\measure\lib\multiprocessing\reduction.py”，第60行，在转储中 ForkingPickler(文件、协议).转储(Obj) AttributeError：无法选择本地对象‘get_DataSet.DataSet’

当我在自己的数据集上进行训练时，会出现一个错误。有人能帮我什么忙吗？

你好，我也遇到了这个问题，请问你解决了吗

[Uio96]

Thanks for adopting this code for multi-processing. Pytorch multi-processing requires a fixed dataset class, while in our case the dataset class is created dynamically here. A ugly but simple way to do this is to manually and explicitly create the specific combination like:

class Dataset(dataset_factory['coco'], _sample_factory['ctdet']):
  pass

def get_dataset(dataset, task):
  return Dataset

@ZHANG-hengzhi The author's simple solution did not work on my end. But I solved the problem by explicitly constructing a new class including all the functions and attributes from the dataset and sample classes.

[Jumponthemoon]

his code for

Would you further explain how you do this?

[aneesurhashmi]

I also faced this issue, A quick fix is to set num_workers=0 (default). Apparently this is because collate_fn does not support num_workers > 0

[slightY]

@Uio96 I have tried your method, but there is no difference between single threading and multi threading