数据集处理问题

[Liwiman]

感谢您提出了这么优秀的模型！我在使用您的模型框架对我的数据进行训练的过程中，测试结果为‘forest land’这一类标签他的IoU=0，Acc=0，Fscore=nan，precision=nan，影响了我的测试结果，经过debug发现可能是标签设置过程中forest land这一类标签没有打上去，该如何修改呢

[DotWang]

@Liwiman 您的问题缺乏细节，我无法回答，我不知道你有几类，也不知道你 forest land 是第几类，或者其对应的label index是多少，因此我无法给出修改config的意见

[Liwiman]

感谢您的回答我已经解决了上诉问题，并将模型投入训练了，但我在训练过程中出现了问题，想恳请您解答。 我训练的数据集是128*128的100000张遥感图片，在 预训练权重的基础上进行训练，设置batch_size为16训练107500个iter后设置batch_size为40训练了22500个iter，但是模型效果很差（val的图片也是训练的图片，其实是在训练集的部分上val）：{"mode": "val", "epoch": 7, "iter": 5129, "lr": 5e-05, "aAcc": 0.6659, "mIoU": 0.4843, "mAcc": 0.6469, "mFscore": 0.6483, "mPrecision": 0.6688, "mRecall": 0.6469, "IoU.water": 0.5942, "IoU.transportation": 0.4205, "IoU.architecture": 0.5802, "IoU.farmland": 0.4988, "IoU.grassland": 0.3906, "IoU.forest land": 0.5741, "IoU.bare soil": 0.3818, "IoU.others": 0.4341, "Acc.water": 0.855, "Acc.transportation": 0.5327, "Acc.architecture": 0.6978, "Acc.farmland": 0.6573, "Acc.grassland": 0.6929, "Acc.forest land": 0.6918, "Acc.bare soil": 0.4493, "Acc.others": 0.5983, "Fscore.water": 0.7455, "Fscore.transportation": 0.592, "Fscore.architecture": 0.7343, "Fscore.farmland": 0.6656, "Fscore.grassland": 0.5617, "Fscore.forest land": 0.7294, "Fscore.bare soil": 0.5526, "Fscore.others": 0.6054, "Precision.water": 0.6608, "Precision.transportation": 0.6662, "Precision.architecture": 0.7749, "Precision.farmland": 0.6741, "Precision.grassland": 0.4723, "Precision.forest land": 0.7715, "Precision.bare soil": 0.7175, "Precision.others": 0.6128, "Recall.water": 0.855, "Recall.transportation": 0.5327, "Recall.architecture": 0.6978, "Recall.farmland": 0.6573, "Recall.grassland": 0.6929, "Recall.forest land": 0.6918, "Recall.bare soil": 0.4493, "Recall.others": 0.5983}，就算输入的是训练集图片，推断效果还是很差： 而该图片的label可视化应该是这样的： 我模型的config文件如下： norm_cfg = dict(type='BN', requires_grad=True) model = dict( type='EncoderDecoder', pretrained= 'work_dirs/cd_train/iter_25000.pth', backbone=dict( type='ViTAE_Window_NoShift_basic', RC_tokens_type=['swin', 'swin', 'transformer', 'transformer'], NC_tokens_type=['swin', 'swin', 'transformer', 'transformer'], stages=4, embed_dims=[64, 64, 128, 256], token_dims=[64, 128, 256, 512], downsample_ratios=[4, 2, 2, 2], NC_depth=[2, 2, 8, 2], NC_heads=[1, 2, 4, 8], RC_heads=[1, 1, 2, 4], mlp_ratio=4.0, NC_group=[1, 32, 64, 128], RC_group=[1, 16, 32, 64], img_size=512, window_size=7, drop_path_rate=0.3), decode_head=dict( type='UPerHead', in_channels=[64, 128, 256, 512], in_index=[0, 1, 2, 3], pool_scales=(1, 2, 3, 6), channels=512, dropout_ratio=0.1, num_classes=8, norm_cfg=dict(type='BN', requires_grad=True), align_corners=False, loss_decode=dict( type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0), ignore_index=8), auxiliary_head=dict( type='FCNHead', in_channels=256, in_index=2, channels=256, num_convs=1, concat_input=False, dropout_ratio=0.1, num_classes=8, norm_cfg=dict(type='BN', requires_grad=True), align_corners=False, loss_decode=dict( type='CrossEntropyLoss', use_sigmoid=False, loss_weight=0.4), ignore_index=8), train_cfg=dict(), test_cfg=dict(mode='whole')) dataset_type = 'HuaweiDataset' data_root = '/output/mmsegmentation/small_train' img_norm_cfg = dict( mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True) crop_size = (512, 512) train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(type='Resize', img_scale=(512, 512), ratio_range=(1.0, 1.0)), dict(type='RandomFlip', flip_ratio=0.5), dict(type='PhotoMetricDistortion'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='Pad', size=(512, 512), pad_val=0, seg_pad_val=255), dict(type='DefaultFormatBundle'), dict(type='Collect', keys=['img', 'gt_semantic_seg']) ] val_pipeline = [ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(512, 512), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='RandomFlip'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ] test_pipeline = [ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(256, 256), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='Resize', img_scale=(512, 512)), dict(type='RandomFlip'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ] data = dict( samples_per_gpu=16, workers_per_gpu=8, train=dict( type='HuaweiDataset', data_root='/output/mmsegmentation/small_train', img_dir='images', ann_dir='labels', pipeline=[ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(type='Resize', img_scale=(512, 512), ratio_range=(1.0, 1.0)), dict(type='RandomFlip', flip_ratio=0.5), dict(type='PhotoMetricDistortion'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='Pad', size=(512, 512), pad_val=0, seg_pad_val=255), dict(type='DefaultFormatBundle'), dict(type='Collect', keys=['img', 'gt_semantic_seg']) ], split='splits/train.txt'), val=dict( type='HuaweiDataset', data_root='/output/mmsegmentation/small_train', img_dir='images', ann_dir='anns', pipeline=[ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(256, 256), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='Resize', img_scale=(512, 512)), dict(type='RandomFlip'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ], split='splits/val.txt'), test=dict( type='HuaweiDataset', data_root='/output/mmsegmentation/small_train', img_dir='images', ann_dir='anns', pipeline=[ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(256, 256), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='Resize', img_scale=(512, 512)), dict(type='RandomFlip'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ], split='splits/val.txt')) log_config = dict( interval=50, hooks=[dict(type='TextLoggerHook', by_epoch=False)]) dist_params = dict(backend='nccl') log_level = 'INFO' load_from = '/home/mmsegmentation/work_dirs/cd_train/iter_25000.pth' resume_from = '/home/mmsegmentation/work_dirs/cd_train/iter_25000.pth' workflow = [('train', 1)] cudnn_benchmark = True optimizer = dict( type='AdamW', lr=6e-05, betas=(0.9, 0.999), weight_decay=0.01, paramwise_cfg=dict( custom_keys=dict( absolute_pos_embed=dict(decay_mult=0.0), relative_position_bias_table=dict(decay_mult=0.0), norm=dict(decay_mult=0.0)))) optimizer_config = dict() lr_config = dict( policy='poly', warmup='linear', warmup_iters=1500, warmup_ratio=1e-06, power=1.0, min_lr=0.0, by_epoch=False) runner = dict(type='IterBasedRunner', max_iters=10000) checkpoint_config = dict(by_epoch=False, interval=1000) evaluation = dict(interval=1000, metric=['mIoU', 'mFscore'], pre_eval=True) find_unused_parameters = True work_dir = './work_dirs/cd_train' gpu_ids = range(0, 1) auto_resume = False seed = 0 device = 'cuda'

训练是采用tools/train.py这个文件进行训练的，您能帮我解答为什么训练效果不好吗？万分感谢！

[DotWang]

@Liwiman 从你给的图看，大概率是标签没对齐，标签和类别的配置需要针对不同数据集进行专门设置，您可以参考以下issue

9

21

23