Open amaanda opened 3 years ago
@amaanda Thanks for interests. Maybe num_classes should be 2, including objects and background.
thanks for the quick reply! =) hm i see. my dataset contains only one class, do you consider background as a class inside your code or should i edit my json to include this class?
@amaanda Usually, donot need to change json. Just ensure that the gt class is one and background is zero during training.
ok. thanks once again!
@JialeCao001 Hey I am having the same issue but I my class number seems to be correct and the json seems file. My config:
# model settings
model = dict(
type='SipMask',
pretrained='open-mmlab://resnet50_caffe',
backbone=dict(
type='ResNet',
depth=50,
num_stages=4,
out_indices=(0, 1, 2, 3),
frozen_stages=1,
norm_cfg=dict(type='BN', requires_grad=False),
style='caffe'),
neck=dict(
type='FPN',
in_channels=[256, 512, 1024, 2048],
out_channels=256,
start_level=1,
add_extra_convs=True,
extra_convs_on_inputs=False, # use P5
num_outs=5,
relu_before_extra_convs=True),
bbox_head=dict(
type='SipMaskHead',
num_classes=2,
in_channels=256,
stacked_convs=3,
feat_channels=256,
strides=[8, 16, 32, 64, 128],
loss_cls=dict(
type='FocalLoss',
use_sigmoid=True,
gamma=2.0,
alpha=0.25,
loss_weight=1.0),
loss_bbox=dict(type='IoULoss', loss_weight=1.0),
loss_centerness=dict(
type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
center_sampling=True,
center_sample_radius=1.5))
# training and testing settings
train_cfg = dict(
assigner=dict(
type='MaxIoUAssigner',
pos_iou_thr=0.5,
neg_iou_thr=0.4,
min_pos_iou=0,
ignore_iof_thr=-1),
allowed_border=-1,
pos_weight=-1,
debug=False)
test_cfg = dict(
nms_pre=200,
min_bbox_size=0,
score_thr=0.03,
nms=dict(type='nms', iou_thr=0.5),
max_per_img=10)
# dataset settings
dataset_type = 'CocoDataset'
data_root = './dateset/blob/'
img_norm_cfg = dict(
mean=[102.9801, 115.9465, 122.7717], std=[1.0, 1.0, 1.0], to_rgb=False)
data = dict(
imgs_per_gpu=8,
workers_per_gpu=2,
train=dict(
type=dataset_type,
ann_file=data_root + 'train/annotations.json_coco.json',
img_prefix=data_root + 'train/',
img_scale=(1200, 1200),
img_norm_cfg=img_norm_cfg,
size_divisor=32,
flip_ratio=0.5,
with_mask=True,
with_crowd=False,
with_label=True,
with_track=True
),
val=dict(
type=dataset_type,
ann_file=data_root + 'valid/annotations.json_coco.json',
img_prefix=data_root + 'valid/',
img_scale=(1200, 1200),
img_norm_cfg=img_norm_cfg,
size_divisor=32,
flip_ratio=0,
with_mask=True,
with_crowd=False,
with_label=True
),
test=dict(
type=dataset_type,
ann_file=data_root + 'test/annotations.json_coco.json',
img_prefix=data_root + 'test/',
img_scale=(1200, 1200),
img_norm_cfg=img_norm_cfg,
size_divisor=32,
flip_ratio=0,
with_mask=False,
with_label=False,
test_mode=True
)
)
# optimizer
optimizer = dict(
type='SGD',
lr=0.005,
momentum=0.9,
weight_decay=0.0001,
paramwise_options=dict(bias_lr_mult=2., bias_decay_mult=0.))
optimizer_config = dict(grad_clip=None)
# learning policy
lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=1000,
warmup_ratio=1.0 / 80,
step=[8, 11])
checkpoint_config = dict(interval=1)
# yapf:disable
log_config = dict(
interval=50,
hooks=[
dict(type='TextLoggerHook'),
# dict(type='TensorboardLoggerHook')
])
# yapf:enable
# runtime settings
total_epochs = 12
device_ids = [0]
#dist_params = dict(backend='nccl')
log_level = 'INFO'
work_dir = './dataset/blob/result/'
load_from = './models/vis_sipmask_ms_1x_final.pth'
resume_from = None
workflow = [('train', 1)]
I am tryinng on a single gpu:
{
"name": "Train model",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}\\tools\\train.py",
"console": "integratedTerminal",
"args": [
"configs\\sipmask\\sipmask_r50_caffe_fpn_gn_ms_1x_blob.py",
"--validate",
"--launcher=none"
]
},
Output:
(sip-mask-test) PS D:\Repos\SipMask\SipMask-VIS> python tools\train.py configs\sipmask\sipmask_r50_caffe_fpn_gn_ms_1x_blob.py --validate --launcher=none
2020-11-19 15:06:37,023 - INFO - Distributed training: False
2020-11-19 15:06:37,359 - INFO - load model from: open-mmlab://resnet50_caffe
2020-11-19 15:06:37,436 - WARNING - unexpected key in source state_dict: conv1.bias, layer1.0.downsample.0.bias, layer1.0.conv1.bias, layer1.0.conv2.bias, layer1.0.conv3.bias, layer1.1.conv1.bias, layer1.1.conv2.bias, layer1.1.conv3.bias, layer1.2.conv1.bias, layer1.2.conv2.bias, layer1.2.conv3.bias, layer2.0.downsample.0.bias, layer2.0.conv1.bias, layer2.0.conv2.bias, layer2.0.conv3.bias, layer2.1.conv1.bias, layer2.1.conv2.bias, layer2.1.conv3.bias, layer2.2.conv1.bias, layer2.2.conv2.bias, layer2.2.conv3.bias, layer2.3.conv1.bias, layer2.3.conv2.bias, layer2.3.conv3.bias, layer3.0.downsample.0.bias, layer3.0.conv1.bias, layer3.0.conv2.bias, layer3.0.conv3.bias, layer3.1.conv1.bias, layer3.1.conv2.bias, layer3.1.conv3.bias, layer3.2.conv1.bias, layer3.2.conv2.bias, layer3.2.conv3.bias, layer3.3.conv1.bias, layer3.3.conv2.bias, layer3.3.conv3.bias, layer3.4.conv1.bias, layer3.4.conv2.bias, layer3.4.conv3.bias, layer3.5.conv1.bias, layer3.5.conv2.bias, layer3.5.conv3.bias, layer4.0.downsample.0.bias, layer4.0.conv1.bias, layer4.0.conv2.bias, layer4.0.conv3.bias, layer4.1.conv1.bias, layer4.1.conv2.bias, layer4.1.conv3.bias, layer4.2.conv1.bias, layer4.2.conv2.bias, layer4.2.conv3.bias
missing keys in source state_dict: layer2.0.bn1.num_batches_tracked, layer3.0.downsample.1.num_batches_tracked, layer2.1.bn3.num_batches_tracked, layer1.0.bn3.num_batches_tracked, bn1.num_batches_tracked, layer1.2.bn2.num_batches_tracked, layer2.2.bn3.num_batches_tracked, layer3.4.bn2.num_batches_tracked, layer1.0.downsample.1.num_batches_tracked, layer4.0.downsample.1.num_batches_tracked, layer3.3.bn1.num_batches_tracked, layer3.4.bn3.num_batches_tracked, layer3.4.bn1.num_batches_tracked, layer3.2.bn2.num_batches_tracked, layer4.2.bn2.num_batches_tracked, layer2.1.bn1.num_batches_tracked, layer1.2.bn3.num_batches_tracked, layer2.0.downsample.1.num_batches_tracked, layer3.5.bn1.num_batches_tracked, layer2.0.bn2.num_batches_tracked, layer3.1.bn2.num_batches_tracked, layer2.2.bn2.num_batches_tracked, layer3.0.bn2.num_batches_tracked, layer4.1.bn3.num_batches_tracked, layer3.0.bn1.num_batches_tracked, layer2.1.bn2.num_batches_tracked, layer1.0.bn2.num_batches_tracked, layer4.0.bn1.num_batches_tracked, layer4.1.bn1.num_batches_tracked, layer2.3.bn2.num_batches_tracked, layer1.1.bn2.num_batches_tracked, layer3.2.bn3.num_batches_tracked, layer3.0.bn3.num_batches_tracked, layer4.0.bn2.num_batches_tracked, layer4.2.bn1.num_batches_tracked, layer2.3.bn3.num_batches_tracked, layer1.1.bn1.num_batches_tracked, layer4.2.bn3.num_batches_tracked, layer2.2.bn1.num_batches_tracked, layer4.0.bn3.num_batches_tracked, layer3.1.bn3.num_batches_tracked, layer2.0.bn3.num_batches_tracked, layer1.1.bn3.num_batches_tracked, layer1.2.bn1.num_batches_tracked, layer3.1.bn1.num_batches_tracked, layer3.2.bn1.num_batches_tracked, layer3.5.bn3.num_batches_tracked, layer4.1.bn2.num_batches_tracked, layer3.5.bn2.num_batches_tracked, layer3.3.bn3.num_batches_tracked, layer3.3.bn2.num_batches_tracked, layer2.3.bn1.num_batches_tracked, layer1.0.bn1.num_batches_tracked
Traceback (most recent call last):
File "tools\train.py", line 110, in <module>
main()
File "tools\train.py", line 88, in main
datasets = [build_dataset(cfg.data.train)]
File "D:\Repos\SipMask\SipMask-VIS\tools\..\mmdet\datasets\builder.py", line 39, in build_dataset
dataset = build_from_cfg(cfg, DATASETS, default_args)
File "D:\Repos\SipMask\SipMask-VIS\tools\..\mmdet\utils\registry.py", line 74, in build_from_cfg
return obj_type(**args)
TypeError: __init__() got an unexpected keyword argument 'with_track'
(sip-mask-test) PS D:\Repos\SipMask\SipMask-VIS>
What am I doing wrong? Thanks in advance.
(sip-mask-test) PS D:\Repos\SipMask\SipMask-VIS> python tools\train.py configs\sipmask\sipmask_r50_caffe_fpn_gn_ms_1x_blob.py --validate --launcher=none
2020-11-19 16:23:45,230 - INFO - Distributed training: False
2020-11-19 16:23:45,518 - INFO - load model from: open-mmlab://resnet50_caffe
2020-11-19 16:23:45,585 - WARNING - unexpected key in source state_dict: conv1.bias, layer1.0.downsample.0.bias, layer1.0.conv1.bias, layer1.0.conv2.bias, layer1.0.conv3.bias, layer1.1.conv1.bias, layer1.1.conv2.bias, layer1.1.conv3.bias, layer1.2.conv1.bias, layer1.2.conv2.bias, layer1.2.conv3.bias, layer2.0.downsample.0.bias, layer2.0.conv1.bias, layer2.0.conv2.bias, layer2.0.conv3.bias, layer2.1.conv1.bias, layer2.1.conv2.bias, layer2.1.conv3.bias, layer2.2.conv1.bias, layer2.2.conv2.bias, layer2.2.conv3.bias, layer2.3.conv1.bias, layer2.3.conv2.bias, layer2.3.conv3.bias, layer3.0.downsample.0.bias, layer3.0.conv1.bias, layer3.0.conv2.bias, layer3.0.conv3.bias, layer3.1.conv1.bias, layer3.1.conv2.bias, layer3.1.conv3.bias, layer3.2.conv1.bias, layer3.2.conv2.bias, layer3.2.conv3.bias, layer3.3.conv1.bias, layer3.3.conv2.bias, layer3.3.conv3.bias, layer3.4.conv1.bias, layer3.4.conv2.bias, layer3.4.conv3.bias, layer3.5.conv1.bias, layer3.5.conv2.bias, layer3.5.conv3.bias, layer4.0.downsample.0.bias, layer4.0.conv1.bias, layer4.0.conv2.bias, layer4.0.conv3.bias, layer4.1.conv1.bias, layer4.1.conv2.bias, layer4.1.conv3.bias, layer4.2.conv1.bias, layer4.2.conv2.bias, layer4.2.conv3.bias
missing keys in source state_dict: layer1.1.bn2.num_batches_tracked, layer3.5.bn2.num_batches_tracked, layer4.1.bn2.num_batches_tracked, layer1.0.downsample.1.num_batches_tracked, layer3.0.downsample.1.num_batches_tracked, layer1.2.bn2.num_batches_tracked, layer3.5.bn3.num_batches_tracked, layer3.3.bn1.num_batches_tracked, layer2.3.bn1.num_batches_tracked, layer2.0.bn3.num_batches_tracked, layer1.1.bn1.num_batches_tracked, bn1.num_batches_tracked, layer4.0.downsample.1.num_batches_tracked, layer4.2.bn3.num_batches_tracked, layer1.0.bn2.num_batches_tracked, layer3.2.bn2.num_batches_tracked, layer2.2.bn2.num_batches_tracked, layer3.0.bn3.num_batches_tracked, layer3.0.bn1.num_batches_tracked, layer1.0.bn3.num_batches_tracked, layer2.2.bn1.num_batches_tracked, layer4.2.bn1.num_batches_tracked, layer1.0.bn1.num_batches_tracked, layer3.4.bn1.num_batches_tracked, layer1.1.bn3.num_batches_tracked, layer4.2.bn2.num_batches_tracked, layer3.4.bn3.num_batches_tracked, layer3.1.bn2.num_batches_tracked, layer3.2.bn1.num_batches_tracked, layer1.2.bn3.num_batches_tracked, layer2.0.downsample.1.num_batches_tracked, layer3.5.bn1.num_batches_tracked, layer3.0.bn2.num_batches_tracked, layer4.1.bn3.num_batches_tracked, layer2.0.bn1.num_batches_tracked, layer3.1.bn1.num_batches_tracked, layer2.1.bn2.num_batches_tracked, layer3.3.bn3.num_batches_tracked, layer4.1.bn1.num_batches_tracked, layer4.0.bn3.num_batches_tracked, layer1.2.bn1.num_batches_tracked, layer3.1.bn3.num_batches_tracked, layer3.2.bn3.num_batches_tracked, layer2.1.bn3.num_batches_tracked, layer4.0.bn1.num_batches_tracked, layer3.3.bn2.num_batches_tracked, layer4.0.bn2.num_batches_tracked, layer2.0.bn2.num_batches_tracked, layer2.3.bn3.num_batches_tracked, layer2.2.bn3.num_batches_tracked, layer3.4.bn2.num_batches_tracked, layer2.1.bn1.num_batches_tracked, layer2.3.bn2.num_batches_tracked
loading annotations into memory...
Done (t=0.02s)
creating index...
index created!
Starting non distributed training
2020-11-19 16:23:47,221 - INFO - Start running, host: hamba@DESKTOP-AV82M29, work_dir: D:\Repos\SipMask\SipMask-VIS\dataset\blob\result
2020-11-19 16:23:47,222 - INFO - workflow: [('train', 1)], max: 12 epochs
Traceback (most recent call last):
File "tools\train.py", line 110, in <module>
main()
File "tools\train.py", line 106, in main
logger=logger)
File "D:\Repos\SipMask\SipMask-VIS\tools\..\mmdet\apis\train.py", line 61, in train_detector
_non_dist_train(model, dataset, cfg, validate=validate)
File "D:\Repos\SipMask\SipMask-VIS\tools\..\mmdet\apis\train.py", line 222, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "D:\ProgramData\Miniconda3\envs\sip-mask-test\lib\site-packages\mmcv\runner\runner.py", line 358, in run
epoch_runner(data_loaders[i], **kwargs)
File "D:\ProgramData\Miniconda3\envs\sip-mask-test\lib\site-packages\mmcv\runner\runner.py", line 260, in train
for i, data_batch in enumerate(data_loader):
File "C:\Users\hamba\AppData\Roaming\Python\Python37\site-packages\torch\utils\data\dataloader.py", line 363, in __next__
data = self._next_data()
File "C:\Users\hamba\AppData\Roaming\Python\Python37\site-packages\torch\utils\data\dataloader.py", line 989, in _next_data
return self._process_data(data)
File "C:\Users\hamba\AppData\Roaming\Python\Python37\site-packages\torch\utils\data\dataloader.py", line 1014, in _process_data
data.reraise()
File "C:\Users\hamba\AppData\Roaming\Python\Python37\site-packages\torch\_utils.py", line 395, in reraise
raise self.exc_type(msg)
Exception: Caught Exception in DataLoader worker process 0.
Original Traceback (most recent call last):
File "C:\Users\hamba\AppData\Roaming\Python\Python37\site-packages\torch\utils\data\_utils\worker.py", line 185, in _worker_loop
data = fetcher.fetch(index)
File "C:\Users\hamba\AppData\Roaming\Python\Python37\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\hamba\AppData\Roaming\Python\Python37\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\Repos\SipMask\SipMask-VIS\tools\..\mmdet\datasets\custom.py", line 181, in __getitem__
data = self.prepare_train_img(idx)
File "D:\Repos\SipMask\SipMask-VIS\tools\..\mmdet\datasets\custom.py", line 215, in prepare_train_img
ann = self.get_ann_info(idx)
File "D:\Repos\SipMask\SipMask-VIS\tools\..\mmdet\datasets\coco.py", line 45, in get_ann_info
return self._parse_ann_info(ann_info, self.with_mask)
File "D:\Repos\SipMask\SipMask-VIS\tools\..\mmdet\datasets\coco.py", line 93, in _parse_ann_info
gt_masks.append(self.coco.annToMask(ann))
File "D:\ProgramData\Miniconda3\envs\sip-mask-test\lib\site-packages\pycocotools\coco.py", line 439, in annToMask
rle = self.annToRLE(ann)
File "D:\ProgramData\Miniconda3\envs\sip-mask-test\lib\site-packages\pycocotools\coco.py", line 424, in annToRLE
rles = maskUtils.frPyObjects(segm, h, w)
File "pycocotools\_mask.pyx", line 307, in pycocotools._mask.frPyObjects
Exception: input type is not supported.
(sip-mask-test) PS D:\Repos\SipMask\SipMask-VIS>
It seems the issue is that my data is in polygon format, is there a way to train with this? If not how can I convert this to RLE?
I managed to convert my data to RLE but I cannot train the modell still.
Exception has occurred: TypeError
forward_train() missing 3 required positional arguments: 'ref_img', 'ref_bboxes', and 'gt_pids'
File "D:\Repos\SipMask\SipMask-VIS\mmdet\models\detectors\base.py", line 86, in forward
return self.forward_train(img, img_meta, **kwargs)
File "D:\Repos\SipMask\SipMask-VIS\mmdet\core\fp16\decorators.py", line 49, in new_func
return old_func(*args, **kwargs)
File "D:\Repos\SipMask\SipMask-VIS\mmdet\apis\train.py", line 38, in batch_processor
losses = model(**data)
File "D:\Repos\SipMask\SipMask-VIS\mmdet\apis\train.py", line 221, in _non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "D:\Repos\SipMask\SipMask-VIS\mmdet\apis\train.py", line 60, in train_detector
_non_dist_train(model, dataset, cfg, validate=validate)
File "D:\Repos\SipMask\SipMask-VIS\tools\train.py", line 106, in main
logger=logger)
File "D:\Repos\SipMask\SipMask-VIS\tools\train.py", line 110, in <module>
main()
@JialeCao001 Any ideas?
@Traderain I cannot get your point. What is your final error?
I managed to convert my data to RLE but I cannot train the modell still.
Exception has occurred: TypeError forward_train() missing 3 required positional arguments: 'ref_img', 'ref_bboxes', and 'gt_pids' File "D:\Repos\SipMask\SipMask-VIS\mmdet\models\detectors\base.py", line 86, in forward return self.forward_train(img, img_meta, **kwargs) File "D:\Repos\SipMask\SipMask-VIS\mmdet\core\fp16\decorators.py", line 49, in new_func return old_func(*args, **kwargs) File "D:\Repos\SipMask\SipMask-VIS\mmdet\apis\train.py", line 38, in batch_processor losses = model(**data) File "D:\Repos\SipMask\SipMask-VIS\mmdet\apis\train.py", line 221, in _non_dist_train runner.run(data_loaders, cfg.workflow, cfg.total_epochs) File "D:\Repos\SipMask\SipMask-VIS\mmdet\apis\train.py", line 60, in train_detector _non_dist_train(model, dataset, cfg, validate=validate) File "D:\Repos\SipMask\SipMask-VIS\tools\train.py", line 106, in main logger=logger) File "D:\Repos\SipMask\SipMask-VIS\tools\train.py", line 110, in <module> main()
@JialeCao001 Any ideas?
@JialeCao001 This is the final issue. My annotations are ms coco (rle)
@Traderain dataset_type = 'CocoDataset' should be changed to dataset_type = 'YTVOSDataset'
So this cant be trained with ms coco?
On 2020. Dec 4., Fri at 2:56, Jiale Cao notifications@github.com wrote:
@Traderain https://github.com/Traderain dataset_type = 'CocoDataset' should be changed to dataset_type = 'YTVOSDataset'
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JialeCao001/SipMask/issues/26#issuecomment-738504604, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQ5LVFAN3BOJW4IHSQRGVTSTA6UVANCNFSM4TRBJVMA .
@Traderain For image instance segmentation, you can use coco-style annotations with SipMask-benchmark or SipMask-mmdet. For video instance segmentation, Sipmask-VIS only supports the yotube-vis style annotations.
hi! thanks for sharing your work :)
i am trying to train using a custom dataset, with only 1 class. i keep getting the following error, i believe it's because the number of classes is different from coco:
i changed the config file with the path to my dataset, which is in coco format.
should i do something else or are this changes enough? apparently the training is running fine.
thanks in advance!