facebookresearch / detectron2

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
https://detectron2.readthedocs.io/en/latest/
Apache License 2.0
30.55k stars 7.49k forks source link

KeyError: "Dataset 'm' is not registered! #1946

Closed yangzgtyut closed 4 years ago

yangzgtyut commented 4 years ago

Instructions To Reproduce the 🐛 Bug:

I train a custom data set on two computers at the same time. The models are "COCO-Detection/faster_rcnn_R_50_C4_3x.yaml" and "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml". The code uses the demo on colab, and changes the number of categories and other parameters to adapt to the data set. Towards the end, an error occurred on both computers at the same time: KeyError: "Dataset'm' is not registered!

  1. what changes you made (git diff) or what code you wrote The code follows the demo on colab. The difference is only the dataset:
    from detectron2.data.datasets import register_coco_instances
    register_coco_instances("my_dataset_train1", {}, "dataset/train/train.json", "dataset/train/images")
    register_coco_instances("my_dataset_train2", {}, "dataset/round1/train_no_poly.json", "dataset/round1/restricted")
    register_coco_instances("my_dataset_val", {}, "dataset/val/val.json", "dataset/val/images")
    ...
    cfg.DATASETS.TRAIN = ("my_dataset_train1","my_dataset_train2",)
    cfg.DATASETS.TEST = ("my_dataset_val")
    cfg.MODEL.ROI_HEADS.NUM_CLASSES = 5
  2. what you observed (including full logs): computer1:
    
    [08/23 08:12:30 d2.utils.events]:  eta: 0:00:18  iter: 69979  total_loss: 0.152  loss_cls: 0.025  loss_box_reg: 0.110  loss_rpn_cls: 0.003  loss_rpn_loc: 0.012  time: 0.8694  data_time: 0.0166  lr: 0.000250  max_mem: 7347M
    ERROR [08/23 08:12:48 d2.engine.train_loop]: Exception during training:
    Traceback (most recent call last):
    File "/home/yangzg/project/master/detectron2-master/detectron2/data/catalog.py", line 55, in get
    f = DatasetCatalog._REGISTERED[name]
    KeyError: 'm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yangzg/project/master/detectron2-master/detectron2/engine/train_loop.py", line 133, in train self.after_step() File "/home/yangzg/project/master/detectron2-master/detectron2/engine/train_loop.py", line 154, in after_step h.after_step() File "/home/yangzg/project/master/detectron2-master/detectron2/engine/hooks.py", line 350, in after_step self._do_eval() File "/home/yangzg/project/master/detectron2-master/detectron2/engine/hooks.py", line 324, in _do_eval results = self._func() File "/home/yangzg/project/master/detectron2-master/detectron2/engine/defaults.py", line 353, in test_and_save_results self._last_eval_results = self.test(self.cfg, self.model) File "/home/yangzg/project/master/detectron2-master/detectron2/engine/defaults.py", line 504, in test data_loader = cls.build_test_loader(cfg, dataset_name) File "/home/yangzg/project/master/detectron2-master/detectron2/engine/defaults.py", line 463, in build_test_loader return build_detection_test_loader(cfg, dataset_name) File "/home/yangzg/project/master/detectron2-master/detectron2/data/build.py", line 381, in build_detection_test_loader dataset_dicts = get_detection_dataset_dicts( File "/home/yangzg/project/master/detectron2-master/detectron2/data/build.py", line 225, in get_detection_dataset_dicts dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names] File "/home/yangzg/project/master/detectron2-master/detectron2/data/build.py", line 225, in dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names] File "/home/yangzg/project/master/detectron2-master/detectron2/data/catalog.py", line 57, in get raise KeyError( KeyError: "Dataset 'm' is not registered! Available datasets are: coco_2014_train, coco_2014_val, coco_2014_minival, coco_2014_minival_100, coco_2014_valminusminival, coco_2017_train, coco_2017_val, coco_2017_test, coco_2017_test-dev, coco_2017_val_100, keypoints_coco_2014_train, keypoints_coco_2014_val, keypoints_coco_2014_minival, keypoints_coco_2014_valminusminival, keypoints_coco_2014_minival_100, keypoints_coco_2017_train, keypoints_coco_2017_val, keypoints_coco_2017_val_100, coco_2017_train_panoptic_separated, coco_2017_train_panoptic_stuffonly, coco_2017_val_panoptic_separated, coco_2017_val_panoptic_stuffonly, coco_2017_val_100_panoptic_separated, coco_2017_val_100_panoptic_stuffonly, lvis_v0.5_train, lvis_v0.5_val, lvis_v0.5_val_rand_100, lvis_v0.5_test, lvis_v0.5_train_cocofied, lvis_v0.5_val_cocofied, cityscapes_fine_instance_seg_train, cityscapes_fine_sem_seg_train, cityscapes_fine_instance_seg_val, cityscapes_fine_sem_seg_val, cityscapes_fine_instance_seg_test, cityscapes_fine_sem_seg_test, voc_2007_trainval, voc_2007_train, voc_2007_val, voc_2007_test, voc_2012_trainval, voc_2012_train, voc_2012_val, my_dataset_train1, my_dataset_train2, my_dataset_val" [08/23 08:12:48 d2.engine.hooks]: Overall training speed: 69997 iterations in 16:54:18 (0.8694 s / it) [08/23 08:12:48 d2.engine.hooks]: Total training time: 16:58:13 (0:03:54 on hooks)

KeyError Traceback (most recent call last) ~/project/master/detectron2-master/detectron2/data/catalog.py in get(name) 54 try: ---> 55 f = DatasetCatalog._REGISTERED[name] 56 except KeyError:

KeyError: 'm'

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last) in 2 trainer = DefaultTrainer(cfg) # 训练脚本 3 trainer.resume_or_load(resume=True) # resume=False表示不加载最近的checkpoint ----> 4 trainer.train()

~/project/master/detectron2-master/detectron2/engine/defaults.py in train(self) 400 OrderedDict of results, if evaluation is enabled. Otherwise None. 401 """ --> 402 super().train(self.start_iter, self.max_iter) 403 if len(self.cfg.TEST.EXPECTED_RESULTS) and comm.is_main_process(): 404 assert hasattr(

~/project/master/detectron2-master/detectron2/engine/train_loop.py in train(self, start_iter, max_iter) 131 self.before_step() 132 self.run_step() --> 133 self.after_step() 134 except Exception: 135 logger.exception("Exception during training:")

~/project/master/detectron2-master/detectron2/engine/train_loop.py in after_step(self) 152 def after_step(self): 153 for h in self._hooks: --> 154 h.after_step() 155 # this guarantees, that in each hook's after_step, storage.iter == trainer.iter 156 self.storage.step()

~/project/master/detectron2-master/detectron2/engine/hooks.py in after_step(self) 348 is_final = next_iter == self.trainer.max_iter 349 if is_final or (self._period > 0 and next_iter % self._period == 0): --> 350 self._do_eval() 351 352 def after_train(self):

~/project/master/detectron2-master/detectron2/engine/hooks.py in _do_eval(self) 322 323 def _do_eval(self): --> 324 results = self._func() 325 326 if results:

~/project/master/detectron2-master/detectron2/engine/defaults.py in test_and_save_results() 351 352 def test_and_save_results(): --> 353 self._last_eval_results = self.test(self.cfg, self.model) 354 return self._last_eval_results 355

~/project/master/detectron2-master/detectron2/engine/defaults.py in test(cls, cfg, model, evaluators) 502 results = OrderedDict() 503 for idx, dataset_name in enumerate(cfg.DATASETS.TEST): --> 504 data_loader = cls.build_test_loader(cfg, dataset_name) 505 # When evaluators are passed in as arguments, 506 # implicitly assume that evaluators can be created before data_loader.

~/project/master/detectron2-master/detectron2/engine/defaults.py in build_test_loader(cls, cfg, dataset_name) 461 Overwrite it if you'd like a different data loader. 462 """ --> 463 return build_detection_test_loader(cfg, dataset_name) 464 465 @classmethod

~/project/master/detectron2-master/detectron2/data/build.py in build_detection_test_loader(cfg, dataset_name, mapper) 379 dataset, with test-time transformation and batching. 380 """ --> 381 dataset_dicts = get_detection_dataset_dicts( 382 [dataset_name], 383 filter_empty=False,

~/project/master/detectron2-master/detectron2/data/build.py in get_detection_dataset_dicts(dataset_names, filter_empty, min_keypoints, proposal_files) 223 """ 224 assert len(dataset_names) --> 225 dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names] 226 for dataset_name, dicts in zip(dataset_names, dataset_dicts): 227 assert len(dicts), "Dataset '{}' is empty!".format(dataset_name)

~/project/master/detectron2-master/detectron2/data/build.py in (.0) 223 """ 224 assert len(dataset_names) --> 225 dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names] 226 for dataset_name, dicts in zip(dataset_names, dataset_dicts): 227 assert len(dicts), "Dataset '{}' is empty!".format(dataset_name)

~/project/master/detectron2-master/detectron2/data/catalog.py in get(name) 55 f = DatasetCatalog._REGISTERED[name] 56 except KeyError: ---> 57 raise KeyError( 58 "Dataset '{}' is not registered! Available datasets are: {}".format( 59 name, ", ".join(DatasetCatalog._REGISTERED.keys())

KeyError: "Dataset 'm' is not registered! Available datasets are: coco_2014_train, coco_2014_val, coco_2014_minival, coco_2014_minival_100, coco_2014_valminusminival, coco_2017_train, coco_2017_val, coco_2017_test, coco_2017_test-dev, coco_2017_val_100, keypoints_coco_2014_train, keypoints_coco_2014_val, keypoints_coco_2014_minival, keypoints_coco_2014_valminusminival, keypoints_coco_2014_minival_100, keypoints_coco_2017_train, keypoints_coco_2017_val, keypoints_coco_2017_val_100, coco_2017_train_panoptic_separated, coco_2017_train_panoptic_stuffonly, coco_2017_val_panoptic_separated, coco_2017_val_panoptic_stuffonly, coco_2017_val_100_panoptic_separated, coco_2017_val_100_panoptic_stuffonly, lvis_v0.5_train, lvis_v0.5_val, lvis_v0.5_val_rand_100, lvis_v0.5_test, lvis_v0.5_train_cocofied, lvis_v0.5_val_cocofied, cityscapes_fine_instance_seg_train, cityscapes_fine_sem_seg_train, cityscapes_fine_instance_seg_val, cityscapes_fine_sem_seg_val, cityscapes_fine_instance_seg_test, cityscapes_fine_sem_seg_test, voc_2007_trainval, voc_2007_train, voc_2007_val, voc_2007_test, voc_2012_trainval, voc_2012_train, voc_2012_val, my_dataset_train1, my_dataset_train2, my_dataset_val"

computer2:

[08/23 09:00:22 d2.utils.events]: eta: 0:00:18 iter: 69979 total_loss: 0.2253 loss_cls: 0.05708 loss_box_reg: 0.1523 loss_rpn_cls: 0.0005672 loss_rpn_loc: 0.01882 time: 0.8912 data_time: 0.0289 lr: 0.00025 max_mem: 9167M ERROR [08/23 09:00:41 d2.engine.train_loop]: Exception during training: Traceback (most recent call last): File "/home/yangzg/project/detectron2/detectron2/data/catalog.py", line 51, in get f = self[name] File "/home/yangzg/.conda/envs/yangzg/lib/python3.8/collections/init.py", line 1010, in getitem raise KeyError(key) KeyError: 'm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yangzg/project/detectron2/detectron2/engine/train_loop.py", line 142, in train self.after_step() File "/home/yangzg/project/detectron2/detectron2/engine/train_loop.py", line 163, in after_step h.after_step() File "/home/yangzg/project/detectron2/detectron2/engine/hooks.py", line 349, in after_step self._do_eval() File "/home/yangzg/project/detectron2/detectron2/engine/hooks.py", line 323, in _do_eval results = self._func() File "/home/yangzg/project/detectron2/detectron2/engine/defaults.py", line 355, in test_and_save_results self._last_eval_results = self.test(self.cfg, self.model) File "/home/yangzg/project/detectron2/detectron2/engine/defaults.py", line 504, in test data_loader = cls.build_test_loader(cfg, dataset_name) File "/home/yangzg/project/detectron2/detectron2/engine/defaults.py", line 463, in build_test_loader return build_detection_test_loader(cfg, dataset_name) File "/home/yangzg/project/detectron2/detectron2/data/build.py", line 373, in build_detection_test_loader dataset_dicts = get_detection_dataset_dicts( File "/home/yangzg/project/detectron2/detectron2/data/build.py", line 219, in get_detection_dataset_dicts dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names] File "/home/yangzg/project/detectron2/detectron2/data/build.py", line 219, in dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names] File "/home/yangzg/project/detectron2/detectron2/data/catalog.py", line 53, in get raise KeyError( KeyError: "Dataset 'm' is not registered! Available datasets are: coco_2014_train, coco_2014_val, coco_2014_minival, coco_2014_minival_100, coco_2014_valminusminival, coco_2017_train, coco_2017_val, coco_2017_test, coco_2017_test-dev, coco_2017_val_100, keypoints_coco_2014_train, keypoints_coco_2014_val, keypoints_coco_2014_minival, keypoints_coco_2014_valminusminival, keypoints_coco_2014_minival_100, keypoints_coco_2017_train, keypoints_coco_2017_val, keypoints_coco_2017_val_100, coco_2017_train_panoptic_separated, coco_2017_train_panoptic_stuffonly, coco_2017_val_panoptic_separated, coco_2017_val_panoptic_stuffonly, coco_2017_val_100_panoptic_separated, coco_2017_val_100_panoptic_stuffonly, lvis_v1_train, lvis_v1_val, lvis_v1_test_dev, lvis_v1_test_challenge, lvis_v0.5_train, lvis_v0.5_val, lvis_v0.5_val_rand_100, lvis_v0.5_test, lvis_v0.5_train_cocofied, lvis_v0.5_val_cocofied, cityscapes_fine_instance_seg_train, cityscapes_fine_sem_seg_train, cityscapes_fine_instance_seg_val, cityscapes_fine_sem_seg_val, cityscapes_fine_instance_seg_test, cityscapes_fine_sem_seg_test, voc_2007_trainval, voc_2007_train, voc_2007_val, voc_2007_test, voc_2012_trainval, voc_2012_train, voc_2012_val, my_dataset_train1, my_dataset_train2, my_dataset_val" [08/23 09:00:41 d2.engine.hooks]: Overall training speed: 69997 iterations in 17:19:43 (0.8912 s / it) [08/23 09:00:41 d2.engine.hooks]: Total training time: 17:23:13 (0:03:30 on hooks)

KeyError Traceback (most recent call last) ~/project/detectron2/detectron2/data/catalog.py in get(self, name) 50 try: ---> 51 f = self[name] 52 except KeyError:

~/.conda/envs/yangzg/lib/python3.8/collections/init.py in getitem(self, key) 1009 return self.class.missing(self, key) -> 1010 raise KeyError(key) 1011 def setitem(self, key, item): self.data[key] = item

KeyError: 'm'

During handling of the above exception, another exception occurred:

KeyError Traceback (most recent call last) in 2 trainer = DefaultTrainer(cfg) # 训练脚本 3 trainer.resume_or_load(resume=False) # resume=False表示不加载最近的checkpoint ----> 4 trainer.train()

~/project/detectron2/detectron2/engine/defaults.py in train(self) 400 OrderedDict of results, if evaluation is enabled. Otherwise None. 401 """ --> 402 super().train(self.start_iter, self.max_iter) 403 if len(self.cfg.TEST.EXPECTED_RESULTS) and comm.is_main_process(): 404 assert hasattr(

~/project/detectron2/detectron2/engine/train_loop.py in train(self, start_iter, max_iter) 140 self.before_step() 141 self.run_step() --> 142 self.after_step() 143 except Exception: 144 logger.exception("Exception during training:")

~/project/detectron2/detectron2/engine/train_loop.py in after_step(self) 161 def after_step(self): 162 for h in self._hooks: --> 163 h.after_step() 164 # this guarantees, that in each hook's after_step, storage.iter == trainer.iter 165 self.storage.step()

~/project/detectron2/detectron2/engine/hooks.py in after_step(self) 347 is_final = next_iter == self.trainer.max_iter 348 if is_final or (self._period > 0 and next_iter % self._period == 0): --> 349 self._do_eval() 350 351 def after_train(self):

~/project/detectron2/detectron2/engine/hooks.py in _do_eval(self) 321 322 def _do_eval(self): --> 323 results = self._func() 324 325 if results:

~/project/detectron2/detectron2/engine/defaults.py in test_and_save_results() 353 354 def test_and_save_results(): --> 355 self._last_eval_results = self.test(self.cfg, self.model) 356 return self._last_eval_results 357

~/project/detectron2/detectron2/engine/defaults.py in test(cls, cfg, model, evaluators) 502 results = OrderedDict() 503 for idx, dataset_name in enumerate(cfg.DATASETS.TEST): --> 504 data_loader = cls.build_test_loader(cfg, dataset_name) 505 # When evaluators are passed in as arguments, 506 # implicitly assume that evaluators can be created before data_loader.

~/project/detectron2/detectron2/engine/defaults.py in build_test_loader(cls, cfg, dataset_name) 461 Overwrite it if you'd like a different data loader. 462 """ --> 463 return build_detection_test_loader(cfg, dataset_name) 464 465 @classmethod

~/project/detectron2/detectron2/data/build.py in build_detection_test_loader(cfg, dataset_name, mapper) 371 dataset, with test-time transformation and batching. 372 """ --> 373 dataset_dicts = get_detection_dataset_dicts( 374 [dataset_name], 375 filter_empty=False,

~/project/detectron2/detectron2/data/build.py in get_detection_dataset_dicts(dataset_names, filter_empty, min_keypoints, proposal_files) 217 """ 218 assert len(dataset_names) --> 219 dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names] 220 for dataset_name, dicts in zip(dataset_names, dataset_dicts): 221 assert len(dicts), "Dataset '{}' is empty!".format(dataset_name)

~/project/detectron2/detectron2/data/build.py in (.0) 217 """ 218 assert len(dataset_names) --> 219 dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in dataset_names] 220 for dataset_name, dicts in zip(dataset_names, dataset_dicts): 221 assert len(dicts), "Dataset '{}' is empty!".format(dataset_name)

~/project/detectron2/detectron2/data/catalog.py in get(self, name) 51 f = self[name] 52 except KeyError: ---> 53 raise KeyError( 54 "Dataset '{}' is not registered! Available datasets are: {}".format( 55 name, ", ".join(list(self.keys()))

KeyError: "Dataset 'm' is not registered! Available datasets are: coco_2014_train, coco_2014_val, coco_2014_minival, coco_2014_minival_100, coco_2014_valminusminival, coco_2017_train, coco_2017_val, coco_2017_test, coco_2017_test-dev, coco_2017_val_100, keypoints_coco_2014_train, keypoints_coco_2014_val, keypoints_coco_2014_minival, keypoints_coco_2014_valminusminival, keypoints_coco_2014_minival_100, keypoints_coco_2017_train, keypoints_coco_2017_val, keypoints_coco_2017_val_100, coco_2017_train_panoptic_separated, coco_2017_train_panoptic_stuffonly, coco_2017_val_panoptic_separated, coco_2017_val_panoptic_stuffonly, coco_2017_val_100_panoptic_separated, coco_2017_val_100_panoptic_stuffonly, lvis_v1_train, lvis_v1_val, lvis_v1_test_dev, lvis_v1_test_challenge, lvis_v0.5_train, lvis_v0.5_val, lvis_v0.5_val_rand_100, lvis_v0.5_test, lvis_v0.5_train_cocofied, lvis_v0.5_val_cocofied, cityscapes_fine_instance_seg_train, cityscapes_fine_sem_seg_train, cityscapes_fine_instance_seg_val, cityscapes_fine_sem_seg_val, cityscapes_fine_instance_seg_test, cityscapes_fine_sem_seg_test, voc_2007_trainval, voc_2007_train, voc_2007_val, voc_2007_test, voc_2012_trainval, voc_2012_train, voc_2012_val, my_dataset_train1, my_dataset_train2, my_dataset_val"


## Environment:

computer1:

sys.platform linux Python 3.8.3 (default, May 19 2020, 18:47:26) [GCC 7.3.0] numpy 1.18.1 detectron2 0.1.3 @/home/yangzg/project/master/detectron2-master/detectron2 Compiler GCC 7.5 CUDA compiler CUDA 10.1 detectron2 arch flags sm_75 DETECTRON2_ENV_MODULE PyTorch 1.5.0+cu101 @/home/yangzg/.conda/envs/yangzg/lib/python3.8/site-packages/torch PyTorch debug build False GPU available True GPU 0 GeForce RTX 2080 Ti CUDA_HOME /usr/local/cuda Pillow 7.1.2 torchvision 0.6.0+cu101 @/home/yangzg/.conda/envs/yangzg/lib/python3.8/site-packages/torchvision torchvision arch flags sm_35, sm_50, sm_60, sm_70, sm_75 fvcore 0.1.1.post20200604 cv2 4.2.0


PyTorch built with:

ppwwyyxx commented 4 years ago

cfg.DATASETS.TEST must be a tuple of string.