search

open-mmlab / mmrazor

OpenMMLab Model Compression Toolbox and Benchmark.
https://mmrazor.readthedocs.io/en/latest/
Apache License 2.0
1.47k stars 228 forks source link

ValueError: val_dataloader, val_cfg, and val_evaluator should be either all None or not None, but got val_dataloader=None, val_cfg={'_delete_': True, 'type': 'mmrazor.SingleTeacherDistillValLoop'}, val_evaluator=None #363

Open Xglbrilliant opened 1 year ago

Xglbrilliant commented 1 year ago

Checklist

Describe the question you meet

When I try to experiment with various distillation methods on my own dataset, it reports this error (this is on the wsld distillation method on the dev-1.x branch).Other errors appear when I comment out val_cfg.

python ./tools/train.py \
    configs/distill/mmcls/wsld/wsld_r50_r18_b32_ISIC.py \
    --work-dir work_dirs/wsld_r50_r18_b32_ISIC/

Post related information

  1. The output of pip list | grep "mmcv\|mmrazor\|^torch" 
    mmcv                   2.0.0rc3
mmrazor              1.0.0rc1  /home/s316/workspace/xionggl/mmrazor
torch                    1.12.0
torchaudio           0.12.0
torchvision           0.13.0
  2. Your config file if you modified it or created a new one.
# wsld_r50_r18_b32_ISIC.py
_base_ = [
    '../../../_base_/datasets/mmcls/ISIC_bs32.py',
    '/home/s316/workspace/xionggl/mmclassification/configs/_base_/schedules/ISIC_bs32_lr.py',
    '/home/s316/workspace/xionggl/mmclassification/configs/_base_/default_runtime.py'
]

model = dict(
    _scope_='mmrazor',
    type='SingleTeacherDistill',
    data_preprocessor=dict(
        type='ImgDataPreprocessor',
        # RGB format normalization parameters
        mean=[128.928, 128.928, 128.928],
        std=[64.26, 64.26, 64.26],
        # convert image from BGR to RGB
        bgr_to_rgb=True),
    architecture=dict(
        cfg_path='/home/s316/workspace/xionggl/mmclassification/configs/resnet/resnet18_b32_ISIC.py', pretrained=False),
    teacher=dict(
        cfg_path='/home/s316/workspace/xionggl/mmclassification/configs/resnet/resnet50_b32_ISIC.py', pretrained=True),
    teacher_ckpt='/home/s316/workspace/xionggl/mmclassification/work_dirs/ISIC/resnet50_b32_ISIC_lr-2_result3/best_accuracy_top-1_epoch_31.pth',
    distiller=dict(
        type='ConfigurableDistiller',
        student_recorders=dict(
            fc=dict(type='ModuleOutputs', source='head.fc'),
            gt_labels=dict(type='ModuleInputs', source='head.loss_module')),
        teacher_recorders=dict(
            fc=dict(type='ModuleOutputs', source='head.fc')),
        distill_losses=dict(
            loss_wsld=dict(type='WSLD', tau=2, loss_weight=2.5)),
        loss_forward_mappings=dict(
            loss_wsld=dict(
                student=dict(recorder='fc', from_student=True),
                teacher=dict(recorder='fc', from_student=False),
                gt_labels=dict(
                    recorder='gt_labels', from_student=True, data_idx=1)))))

find_unused_parameters = True

val_cfg = dict(_delete_=True, type='mmrazor.SingleTeacherDistillValLoop')
  1. Your train log file if you meet the problem during training. 
    Traceback (most recent call last):
File "./tools/train.py", line 121, in <module>
main()
File "./tools/train.py", line 114, in main
runner = Runner.from_cfg(cfg)
File "/home/s316/miniconda3/envs/razor/lib/python3.8/site-packages/mmengine/runner/runner.py", line 437, in from_cfg
runner = cls(
File "/home/s316/miniconda3/envs/razor/lib/python3.8/site-packages/mmengine/runner/runner.py", line 316, in __init__
raise ValueError(
ValueError: val_dataloader, val_cfg, and val_evaluator should be either all None or not None, but got val_dataloader=None, val_cfg={'_delete_': True, 'type': 'mmrazor.SingleTeacherDistillValLoop'}, val_evaluator=None
  2. Other code you modified in the mmrazor folder. No
HIT-cwh commented 1 year ago

Hi! Could you provide the codes in ../../../_base_/datasets/mmcls/ISIC_bs32.py. It seems that val_dataloader and val_evaluator are set to None or not assigned. The corresponding code in mmcls is:

val_dataloader = dict(
    batch_size=16,
    num_workers=2,
    dataset=dict(
        type=dataset_type,
        data_prefix='data/cifar10/',
        test_mode=True,
        pipeline=test_pipeline),
    sampler=dict(type='DefaultSampler', shuffle=False),
    persistent_workers=True,
)
val_evaluator = dict(type='Accuracy', topk=(1, ))
Xglbrilliant commented 1 year ago

Hi! Could you provide the codes in ../../../_base_/datasets/mmcls/ISIC_bs32.py. It seems that val_dataloader and val_evaluator are set to None or not assigned. The corresponding code in mmcls is:

val_dataloader = dict(
    batch_size=16,
    num_workers=2,
    dataset=dict(
        type=dataset_type,
        data_prefix='data/cifar10/',
        test_mode=True,
        pipeline=test_pipeline),
    sampler=dict(type='DefaultSampler', shuffle=False),
    persistent_workers=True,
)
val_evaluator = dict(type='Accuracy', topk=(1, ))

First of all thanks for your answer! I tried and modified the config file according to your method, but it seems that a new problem has appeared:

Traceback (most recent call last):
  File "./tools/train.py", line 121, in <module>
    main()
  File "./tools/train.py", line 114, in main
    runner = Runner.from_cfg(cfg)
  File "/home/s316/miniconda3/envs/razor/lib/python3.8/site-packages/mmengine/runner/runner.py", line 437, in from_cfg
    runner = cls(
  File "/home/s316/miniconda3/envs/razor/lib/python3.8/site-packages/mmengine/runner/runner.py", line 346, in __init__
    self.setup_env(env_cfg)
  File "/home/s316/miniconda3/envs/razor/lib/python3.8/site-packages/mmengine/runner/runner.py", line 641, in setup_env
    if env_cfg.get('cudnn_benchmark'):
AttributeError: 'NoneType' object has no attribute 'get'

Here is my ISIC_bs32.py file along with the modified config file:

# ISIC_bs32.py
dataset_type = 'ISIC'
img_norm_cfg = dict(
    mean=[194.53, 139.5, 145.72], std=[36.036, 39.117, 43.55], to_rgb=True)
classes = ['actinic keratosis', 'basal cell carcinoma', 'dermatofibroma', 'melanoma', 'nevus', 'pigmented benign keratosis', 'seborrheic keratosis', 'squamous cell carcinoma', 'vascular lesion']

train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='RandomResizedCrop', size=224),
    dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'),
    dict(type='Normalize', **img_norm_cfg),
    dict(type='ImageToTensor', keys=['img']),
    dict(type='ToTensor', keys=['gt_label']),
    dict(type='Collect', keys=['img', 'gt_label'])
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='Resize', size=(256, -1)),
    dict(type='CenterCrop', crop_size=224),
    dict(type='Normalize', **img_norm_cfg),
    dict(type='ImageToTensor', keys=['img']),
    dict(type='Collect', keys=['img'])
]

train_dataloader = dict(
    batch_size=32,
    num_workers=2,
    dataset=dict(
        type=dataset_type,
        data_prefix='/home/s316/workspace/datasets/ISIC/train/',
        test_mode=False,
        classes=classes,
        pipeline=train_pipeline),
    sampler=dict(type='DefaultSampler', shuffle=True),
    persistent_workers=True,
)

val_dataloader = dict(
    batch_size=32,
    num_workers=2,
    dataset=dict(
        type=dataset_type,
        data_prefix='/home/s316/workspace/datasets/ISIC/test/',
        test_mode=True,
        classes=classes,
        pipeline=test_pipeline),
    sampler=dict(type='DefaultSampler', shuffle=False),
    persistent_workers=True,
)
val_evaluator = dict(type='Accuracy', topk=(1, 5))
_base_ = [
    '../../../_base_/datasets/mmcls/ISIC_bs32.py',
    '/home/s316/workspace/xionggl/mmclassification/configs/_base_/schedules/ISIC_bs32_lr.py',
    '/home/s316/workspace/xionggl/mmclassification/configs/_base_/default_runtime.py'
]

optim_wrapper = dict(
    optimizer=dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0005))
param_scheduler = dict(
    type='MultiStepLR', by_epoch=True, milestones=[30, 60, 90], gamma=0.1)
train_cfg = dict(by_epoch=True, max_epochs=90, val_interval=1)

model = dict(
    _scope_='mmrazor',
    type='SingleTeacherDistill',
    data_preprocessor=dict(
        type='ImgDataPreprocessor',
        mean=[194.53, 139.5, 145.72],
        std=[36.036, 39.117, 43.55],
        bgr_to_rgb=True),
    architecture=dict(
        cfg_path='/home/s316/workspace/xionggl/mmclassification/configs/resnet/resnet18_b32_ISIC.py', pretrained=False),
    teacher=dict(
        cfg_path='/home/s316/workspace/xionggl/mmclassification/configs/resnet/resnet50_b32_ISIC.py', pretrained=True),
    teacher_ckpt='/home/s316/workspace/xionggl/mmclassification/work_dirs/ISIC/resnet50_b32_ISIC_lr-2_result3/best_accuracy_top-1_epoch_31.pth',
    distiller=dict(
        type='ConfigurableDistiller',
        student_recorders=dict(
            fc=dict(type='ModuleOutputs', source='head.fc'),
            gt_labels=dict(type='ModuleInputs', source='head.loss_module')),
        teacher_recorders=dict(
            fc=dict(type='ModuleOutputs', source='head.fc')),
        distill_losses=dict(
            loss_wsld=dict(type='WSLD', tau=2, loss_weight=2.5)),
        loss_forward_mappings=dict(
            loss_wsld=dict(
                student=dict(recorder='fc', from_student=True),
                teacher=dict(recorder='fc', from_student=False),
                gt_labels=dict(
                    recorder='gt_labels', from_student=True, data_idx=1)))))

find_unused_parameters = True

val_cfg = dict(_delete_=True, type='mmrazor.SingleTeacherDistillValLoop')
HIT-cwh commented 1 year ago

Could you provide your codes in /home/s316/workspace/xionggl/mmclassification/configs/_base_/default_runtime.py? It seems that env_cfg is set to None or not assigned. The corresponding code in mmcls is:

env_cfg = dict(
    # whether to enable cudnn benchmark
    cudnn_benchmark=False,
    # set multi process parameters
    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
    # set distributed parameters
    dist_cfg=dict(backend='nccl'),
)

BTW, you can also find your training log in your work_dir and check whether env_cfg is in your Config.