Bug of Segmenter

[Senwang98]

Checklist

1. I have searched related issues but cannot get the expected help.
2. The bug has not been fixed in the latest version.

3. Background I am doing binary seg task. So I tried some SOTA methods such as segformer and segmentor. segformer works well but segmentor failed.

---

Describe the bug Segmentor does not support binary seg task! And the class except background is always 0!

Reproduction

1. What command or script did you run?

_base_ = [
    '../_base_/models/segmenter_vit-b16_mask.py',
    '../_base_/datasets/drive.py', '../_base_/default_runtime.py',
    '../_base_/schedules/schedule_20k.py'
]
optimizer = dict(lr=0.001, weight_decay=0.0)

# img_norm_cfg = dict(
#     mean=[127.5, 127.5, 127.5], std=[127.5, 127.5, 127.5], to_rgb=True)
# crop_size = (512, 512)
# train_pipeline = [
#     dict(type='LoadImageFromFile'),
#     dict(type='LoadAnnotations', reduce_zero_label=True),
#     dict(type='Resize', img_scale=(2048, 512), ratio_range=(0.5, 2.0)),
#     dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.75),
#     dict(type='RandomFlip', prob=0.5),å
#     dict(type='PhotoMetricDistortion'),
#     dict(type='Normalize', **img_norm_cfg),
#     dict(type='Pad', size=crop_size, pad_val=0, seg_pad_val=255),
#     dict(type='DefaultFormatBundle'),
#     dict(type='Collect', keys=['img', 'gt_semantic_seg'])
# ]
# test_pipeline = [
#     dict(type='LoadImageFromFile'),
#     dict(
#         type='MultiScaleFlipAug',
#         img_scale=(2048, 512),
#         # img_ratios=[0.5, 0.75, 1.0, 1.25, 1.5, 1.75],
#         flip=False,
#         transforms=[
#             dict(type='Resize', keep_ratio=True),
#             dict(type='RandomFlip'),
#             dict(type='Normalize', **img_norm_cfg),
#             dict(type='ImageToTensor', keys=['img']),
#             dict(type='Collect', keys=['img'])
#         ])
# ]
# data = dict(
#     # num_gpus: 8 -> batch_size: 8
#     samples_per_gpu=1,
#     train=dict(pipeline=train_pipeline),
#     val=dict(pipeline=test_pipeline),
#     test=dict(pipeline=test_pipeline))

model = dict(
    decode_head=dict(num_classes=2)
)

---

2. Did you make any modifications on the code or config? Did you understand what you have modified? I give the config above. As you can see, I only mofidy the dataset's config path and comment the original segmentor's dataset pipeline. Finally, I modify the decode-head's output class number = 2 instead of 150 in ADE20K.

3. What dataset did you use? Drive

4. How about results?

[MengzhangLI]

segmentor or segmenter? I can't get it.

[Senwang98]

@MengzhangLI It's my fault, it is Segmenter! I found segmenter seems do not supprt DRIVE dataset.

[MengzhangLI]

Ok. It is wired because I just changed num_classes in segmenter unit test and it passed.

[MengzhangLI]

You could keep other settings like learning rate same with original unet drive config and try again.

[Senwang98]

@MengzhangLI Segmenter can train and test successfully, but it seems that the training's result is strange. vessel item is always 0 when evaluation. I have changed num_class in config. I think I should check it twice and thanks for your reply again! I will report my results again later.

[Senwang98]

@MengzhangLI I am back, I find an interesting thing, when using drive dataset as transet for segmenter. dict(type='LoadAnnotations', reduce_zero_label=True), is default setting. In this way, the deocder's num_class = 2 will fail, but num_class=3 is ok which is intuitively reasonable.

I expect I can train segmenter with num_class = 2 will be ok when I set dict(type='LoadAnnotations', reduce_zero_label=False). But in this way, num_class=2 still fail to train segmenter, but num_class=3 is still ok.

[shuaikangma]

@Senwang98 Hello, I met the same problem when I train binary segmentation on segmenter. I want to know whether you change the lable from 0-1 to 1-2 when you change the num_classes. Thanks!

[Senwang98]

@shuaikangma No matter my own dataset or DRIVE dataset, I change the pixel value to 0-1 rather 1-2. I think my mask processing is right, because the DRIVE dataset is processed by MMseg official script!!!

[shuaikangma]

@Senwang98 thank you for your reply. I'll try it

[ilpapds]

Excuse me. I have a similar problem. Do you use use_sigmoid=True or False? Thank you in advance

[Senwang98]

False

[LeopoldACC]

I meet the same issue when I use segmenter to do binary segmentation task,and after I set the

cfg.model.decode_head.num_classes = 3 

the background class is not keeped as zero. So what the reason cause that? I must set the decode num_classes for segmenter specially as below,lol

cfg.model.decode_head.num_classes = 3 if 'segmenter' in model_name else 2

[mengxia1994]

False

want know how did you fix it? I meet the simular problem. I have two classes and i set the num_classes as 3. with pixel label from 0 to 2. It worked well on segformer(ended with a really good performence). But when I try to use segmenter to train, the training looks strange. The decode.acc and loss seems normal, but the val result is nearly always zero.