stop at 4000 iter

[smntjugithub]

2021-12-30 00:46:57,415 - mmdet.ssod - INFO - Iter [3900/180000] lr: 1.000e-03, eta: 1 day, 20:54:50, time: 0.893, data_time: 0.012, memory: 5544, ema_momentum: 0.9990, unsup_loss_rpn_cls: 0.0676, unsup_loss_rpn_bbox: 0.7972, unsup_loss_cls: 0.7338, unsup_acc: 88.3301, unsup_loss_bbox: 0.0259, loss: 1.6245 2021-12-30 00:47:42,376 - mmdet.ssod - INFO - Iter [3950/180000] lr: 1.000e-03, eta: 1 day, 20:53:22, time: 0.899, data_time: 0.013, memory: 5544, ema_momentum: 0.9990, unsup_loss_rpn_cls: 0.0594, unsup_loss_rpn_bbox: 0.8052, unsup_loss_cls: 0.8364, unsup_acc: 88.0898, unsup_loss_bbox: 0.0237, loss: 1.7246 2021-12-30 00:48:26,592 - mmdet.ssod - INFO - Saving checkpoint at 4000 iterations 2021-12-30 00:48:31,781 - mmdet.ssod - INFO - Exp name: soft_teacher_faster_rcnn_r50_caffe_fpn_coco_180k.py 2021-12-30 00:48:31,782 - mmdet.ssod - INFO - Iter [4000/180000] lr: 1.000e-03, eta: 1 day, 20:55:20, time: 0.992, data_time: 0.012, memory: 5544, ema_momentum: 0.9990, unsup_loss_rpn_cls: 0.0693, unsup_loss_rpn_bbox: 0.8008, unsup_loss_cls: 0.6988, unsup_acc: 88.7949, unsup_loss_bbox: 0.0464, loss: 1.6153 [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 3071/3071, 8.7 task/s, elapsed: 355s, ETA: 0sTraceback (most recent call last): File "tools/train.py", line 198, in main() File "tools/train.py", line 193, in main meta=meta, File "/media/smn/Elements SE/SoftTeacher-main/ssod/apis/train.py", line 206, in train_detector runner.run(data_loaders, cfg.workflow) File "/home/smn/anaconda3/envs/Soft/lib/python3.6/site-packages/mmcv/runner/iter_based_runner.py", line 133, in run iter_runner(iter_loaders[i], kwargs) File "/home/smn/anaconda3/envs/Soft/lib/python3.6/site-packages/mmcv/runner/iter_based_runner.py", line 66, in train self.call_hook('after_train_iter') File "/home/smn/anaconda3/envs/Soft/lib/python3.6/site-packages/mmcv/runner/base_runner.py", line 307, in call_hook getattr(hook, fn_name)(self) File "/media/smn/Elements SE/SoftTeacher-main/ssod/utils/hooks/submodules_evaluation.py", line 37, in after_train_iter self._do_evaluate(runner) File "/media/smn/Elements SE/SoftTeacher-main/ssod/utils/hooks/submodules_evaluation.py", line 82, in _do_evaluate key_score = self.evaluate(runner, results, prefix=submodule) File "/media/smn/Elements SE/SoftTeacher-main/ssod/utils/hooks/submodules_evaluation.py", line 110, in evaluate results, logger=runner.logger, self.eval_kwargs File "/home/smn/anaconda3/envs/Soft/lib/python3.6/site-packages/mmdet/datasets/coco.py", line 416, in evaluate result_files, tmp_dir = self.format_results(results, jsonfile_prefix) File "/home/smn/anaconda3/envs/Soft/lib/python3.6/site-packages/mmdet/datasets/coco.py", line 361, in format_results result_files = self.results2json(results, jsonfile_prefix) File "/home/smn/anaconda3/envs/Soft/lib/python3.6/site-packages/mmdet/datasets/coco.py", line 293, in results2json json_results = self._det2json(results) File "/home/smn/anaconda3/envs/Soft/lib/python3.6/site-packages/mmdet/datasets/coco.py", line 230, in _det2json data['category_id'] = self.cat_ids[label] IndexError: list index out of range

[smntjugithub]

IndexError: list index out of range

[MendelXu]

Do you add classes=... to all items in the data? For example, you have 3 classes ['a','b','c']. And you have to change the config like below:

data=dict(
    train=dict(
    ...
   classes= ['a','b','c']
    ),
   val=dict(
   ...
   classes= ['a','b','c']
   ),
   test=dict(
   ...
   classes= ['a','b','c']
  )
)

[smntjugithub]

Thank you very much for your reply. I have made changes as you said, but the training results seem to be abnormal. I have 10 class Can you develop a tutorial that runs on new data? For example, list the configuration parameters that need to be modified. Thank you！

[MendelXu]

Take this config file for example. When you add a new dataset, make sure you have changed two things: 1) the path of images and annotations. 2) the class names of your dataset. Concretely, you have to modify the data dictionary like below:

• Original designed for MS COCO:

  data = dict(
  samples_per_gpu=5,
  workers_per_gpu=5,
  train=dict(
      sup=dict(
          type="CocoDataset",
          ann_file="data/coco/annotations/semi_supervised/instances_train2017.${fold}@${percent}.json",
          img_prefix="data/coco/train2017/",
      ),
      unsup=dict(
          type="CocoDataset",
          ann_file="data/coco/annotations/semi_supervised/instances_train2017.${fold}@${percent}-unlabeled.json",
          img_prefix="data/coco/train2017/",
      ),
  ),
  sampler=dict(
      train=dict(
          sample_ratio=[1, 4],
      )
  ),
  )

• Changed for your new dataset:

  data = dict(
  samples_per_gpu=5,
  workers_per_gpu=5,
  train=dict(
      sup=dict(
          type="CocoDataset",
          ann_file="[YOUR_ANNOTATION_FILE]",
          img_prefix="[YOUR_ANNOTATED_IMAGE_PATH]",
          classes=[YOUR_CLASS_DEFINITIONS],
      ),
      unsup=dict(
          type="CocoDataset",
          ann_file="[YOUR_ANNOTATION_FILE_FOR_UNLABELED_DATA]",
          img_prefix="[YOUR_IMAGE_PATH_FOR_UNLABELED_DATA]",
         classes=[YOUR_CLASS_DEFINITIONS],
      ),
  ),
  val=dict(
          ann_file="[YOUR_ANNOTATION_FILE_FOR_VALIDATION]",
          img_prefix="[YOUR_ANNOTATED_IMAGE_PATH_FOR_VALIDATION]",
          classes=[YOUR_CLASS_DEFINITIONS],
   )
  sampler=dict(
      train=dict(
          sample_ratio=[1, 4],
      )
  ),
  )

[smntjugithub]

I have followed your tutorial to train on my own data set. No errors were reported during the training process. I used two data sets, but it seems that the target detection is not good after training. A large number of useless boxes are generated, and these boxes often correspond to a fixed category, and there are no boxes of other categories. Is it because the number of my categories is much smaller than coco's 80, so some parameters need to be adjusted?

[smntjugithub]

[smntjugithub]

This is the result in another data set. This data set contains 10 types of construction machinery and contains a total of 10,000 pictures. I followed your tutorial for 180k training, and the results are shown in the figure.

[smntjugithub]

This is the result of training on the coco standard data set for 2 hours, and it seems correct. But with my own data set, the above results appeared. I Follow the tutorial you provided exactly. I used two data sets to eliminate the problem of the data set itself.

[MendelXu]

I am not sure whether there are other details I forgot. Could you post your config file here?

[xiangtaowong]

)

hello, I change the config as what u send, but there is still the same error“list index out of range”

[xiangtaowong]

I am not sure whether there are other details I forgot. Could you post your config file here?

and i meet the same question, there are too many bbox in a image in wandb.