distill result

[JennieDD]

在修改了配置文件训练自己的数据后，我得到了teacher的权重文件，运行的代码为：

python tools/mmcls/train_mmcls.py   configs/distill/rkd/rkd_neck_resnet34_resnet18_8xb32_in1k.py   \
--work-dir /data/dataset_81/jd_result/distill_mmcls   \
--cfg-options algorithm.distiller.teacher.init_cfg.type=Pretrained

生成的结果为：

但是出现了一个问题： 生成的经过distill 的权重文件的大小是167MB，但是teacher的权重文件只有163MB，没有起到模型瘦身的效果，这是为什么呢

[humu789]

Please give us your log to analyze it better

[JennieDD]

@humu789 2022-05-06 14:08:22,372 - mmcls - INFO - Environment info:

sys.platform: linux Python: 3.8.8 (default, Feb 24 2021, 21:46:12) [GCC 7.3.0] CUDA available: True GPU 0,1: NVIDIA GeForce RTX 3090 CUDA_HOME: /usr/local/cuda NVCC: Cuda compilation tools, release 11.1, V11.1.105 GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 PyTorch: 1.8.0 PyTorch compiling details: PyTorch built with:

• GCC 7.3
• C++ Version: 201402
• Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 11.1
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
• CuDNN 8.0.5
• Magma 2.5.2
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.9.0 OpenCV: 4.5.5 MMCV: 1.5.0 MMCV Compiler: GCC 7.3 MMCV CUDA Compiler: 11.1 MMClassification: 0.22.0+

2022-05-06 14:08:22,373 - mmcls - INFO - Distributed training: False 2022-05-06 14:08:22,730 - mmcls - INFO - Config: dataset_type = 'ImageNet' img_norm_cfg = dict( mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True) train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='RandomResizedCrop', size=224), dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='ToTensor', keys=['gt_label']), dict(type='Collect', keys=['img', 'gt_label']) ] test_pipeline = [ dict(type='LoadImageFromFile'), dict(type='Resize', size=(256, -1)), dict(type='CenterCrop', crop_size=224), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ] data = dict( samples_per_gpu=32, workers_per_gpu=2, train=dict( type='ImageNet', data_prefix= '/data/dataset_81/cats_dogs_dataset/training_set/training_set', classes='/data/dataset_81/cats_dogs_dataset/classes.txt', pipeline=[ dict(type='LoadImageFromFile'), dict(type='RandomResizedCrop', size=224), dict(type='RandomFlip', flip_prob=0.5, direction='horizontal'), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='ToTensor', keys=['gt_label']), dict(type='Collect', keys=['img', 'gt_label']) ]), val=dict( type='ImageNet', data_prefix='/data/dataset_81/cats_dogs_dataset/val_set/val_set', ann_file='/data/dataset_81/cats_dogs_dataset/val.txt', classes='/data/dataset_81/cats_dogs_dataset/classes.txt', pipeline=[ dict(type='LoadImageFromFile'), dict(type='Resize', size=(256, -1)), dict(type='CenterCrop', crop_size=224), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]), test=dict( type='ImageNet', data_prefix='/data/dataset_81/cats_dogs_dataset/test_set/test_set', ann_file='/data/dataset_81/cats_dogs_dataset/test.txt', classes='/data/dataset_81/cats_dogs_dataset/classes.txt', pipeline=[ dict(type='LoadImageFromFile'), dict(type='Resize', size=(256, -1)), dict(type='CenterCrop', crop_size=224), dict( type='Normalize', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ])) evaluation = dict( interval=1, metric='accuracy', metric_options=dict(topk=(1, ))) optimizer = dict(type='SGD', lr=0.1, momentum=0.9, weight_decay=0.0001) optimizer_config = dict(grad_clip=None) lr_config = dict(policy='step', step=[30, 60, 90]) runner = dict(type='EpochBasedRunner', max_epochs=100) checkpoint_config = dict(interval=1) log_config = dict(interval=100, hooks=[dict(type='TextLoggerHook')]) dist_params = dict(backend='nccl') log_level = 'INFO' load_from = None resume_from = None workflow = [('train', 1)] student = dict( type='mmcls.ImageClassifier', backbone=dict( type='ResNet', depth=18, num_stages=4, out_indices=(3, ), style='pytorch'), neck=dict(type='GlobalAveragePooling'), head=dict( type='LinearClsHead', num_classes=2, in_channels=512, loss=dict(type='CrossEntropyLoss', loss_weight=1.0), topk=(1, ))) teacher_ckpt = '/data/dataset_81/jd_result/epoch_100.pth' teacher = dict( type='mmcls.ImageClassifier', init_cfg=dict( type='Pretrained', checkpoint='/data/dataset_81/jd_result/epoch_100.pth'), backbone=dict( type='ResNet', depth=34, num_stages=4, out_indices=(3, ), style='pytorch'), neck=dict(type='GlobalAveragePooling'), head=dict( type='LinearClsHead', num_classes=2, in_channels=512, loss=dict(type='CrossEntropyLoss', loss_weight=1.0), topk=(1, ))) algorithm = dict( type='GeneralDistill', architecture=dict( type='MMClsArchitecture', model=dict( type='mmcls.ImageClassifier', backbone=dict( type='ResNet', depth=18, num_stages=4, out_indices=(3, ), style='pytorch'), neck=dict(type='GlobalAveragePooling'), head=dict( type='LinearClsHead', num_classes=2, in_channels=512, loss=dict(type='CrossEntropyLoss', loss_weight=1.0), topk=(1, )))), with_student_loss=True, with_teacher_loss=False, distiller=dict( type='SingleTeacherDistiller', teacher=dict( type='mmcls.ImageClassifier', init_cfg=dict( type='Pretrained', checkpoint='/data/dataset_81/jd_result/epoch_100.pth'), backbone=dict( type='ResNet', depth=34, num_stages=4, out_indices=(3, ), style='pytorch'), neck=dict(type='GlobalAveragePooling'), head=dict( type='LinearClsHead', num_classes=2, in_channels=512, loss=dict(type='CrossEntropyLoss', loss_weight=1.0), topk=(1, ))), teacher_trainable=False, teacher_norm_eval=True, components=[ dict( student_module='neck.gap', teacher_module='neck.gap', losses=[ dict( type='DistanceWiseRKD', name='distance_wise_loss', loss_weight=25.0, with_l2_norm=True), dict( type='AngleWiseRKD', name='angle_wise_loss', loss_weight=50.0, with_l2_norm=True) ]) ])) find_unused_parameters = True work_dir = '/data/dataset_81/jd_result/distill_mmcls' auto_resume = False gpu_ids = [0]

2022-05-06 14:08:22,731 - mmcls - INFO - Set random seed to 56855026, deterministic: False 2022-05-06 14:08:23,119 - mmcls - INFO - initialize ResNet with init_cfg [{'type': 'Kaiming', 'layer': ['Conv2d']}, {'type': 'Constant', 'val': 1, 'layer': ['_BatchNorm', 'GroupNorm']}] 2022-05-06 14:08:23,264 - mmcls - INFO - initialize LinearClsHead with init_cfg {'type': 'Normal', 'layer': 'Linear', 'std': 0.01} 2022-05-06 14:08:23,270 - mmcls - INFO - initialize ImageClassifier with init_cfg {'type': 'Pretrained', 'checkpoint': '/data/dataset_81/jd_result/epoch_100.pth'} Name of parameter - Initialization information

architecture.model.backbone.conv1.weight - torch.Size([64, 3, 7, 7]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.bn1.weight - torch.Size([64]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.bn1.bias - torch.Size([64]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer1.0.conv1.weight - torch.Size([64, 64, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer1.0.bn1.weight - torch.Size([64]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer1.0.bn1.bias - torch.Size([64]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer1.0.conv2.weight - torch.Size([64, 64, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer1.0.bn2.weight - torch.Size([64]): Initialized by user-defined init_weights in ResNet

architecture.model.backbone.layer1.0.bn2.bias - torch.Size([64]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer1.1.conv1.weight - torch.Size([64, 64, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer1.1.bn1.weight - torch.Size([64]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer1.1.bn1.bias - torch.Size([64]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer1.1.conv2.weight - torch.Size([64, 64, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer1.1.bn2.weight - torch.Size([64]): Initialized by user-defined init_weights in ResNet

architecture.model.backbone.layer1.1.bn2.bias - torch.Size([64]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer2.0.conv1.weight - torch.Size([128, 64, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer2.0.bn1.weight - torch.Size([128]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer2.0.bn1.bias - torch.Size([128]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer2.0.conv2.weight - torch.Size([128, 128, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer2.0.bn2.weight - torch.Size([128]): Initialized by user-defined init_weights in ResNet

architecture.model.backbone.layer2.0.bn2.bias - torch.Size([128]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer2.0.downsample.0.weight - torch.Size([128, 64, 1, 1]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer2.0.downsample.1.weight - torch.Size([128]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer2.0.downsample.1.bias - torch.Size([128]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer2.1.conv1.weight - torch.Size([128, 128, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer2.1.bn1.weight - torch.Size([128]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer2.1.bn1.bias - torch.Size([128]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer2.1.conv2.weight - torch.Size([128, 128, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer2.1.bn2.weight - torch.Size([128]): Initialized by user-defined init_weights in ResNet

architecture.model.backbone.layer2.1.bn2.bias - torch.Size([128]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer3.0.conv1.weight - torch.Size([256, 128, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer3.0.bn1.weight - torch.Size([256]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer3.0.bn1.bias - torch.Size([256]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer3.0.conv2.weight - torch.Size([256, 256, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer3.0.bn2.weight - torch.Size([256]): Initialized by user-defined init_weights in ResNet

architecture.model.backbone.layer3.0.bn2.bias - torch.Size([256]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer3.0.downsample.0.weight - torch.Size([256, 128, 1, 1]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer3.0.downsample.1.weight - torch.Size([256]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer3.0.downsample.1.bias - torch.Size([256]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer3.1.conv1.weight - torch.Size([256, 256, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer3.1.bn1.weight - torch.Size([256]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer3.1.bn1.bias - torch.Size([256]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer3.1.conv2.weight - torch.Size([256, 256, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer3.1.bn2.weight - torch.Size([256]): Initialized by user-defined init_weights in ResNet

architecture.model.backbone.layer3.1.bn2.bias - torch.Size([256]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer4.0.conv1.weight - torch.Size([512, 256, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer4.0.bn1.weight - torch.Size([512]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer4.0.bn1.bias - torch.Size([512]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer4.0.conv2.weight - torch.Size([512, 512, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer4.0.bn2.weight - torch.Size([512]): Initialized by user-defined init_weights in ResNet

architecture.model.backbone.layer4.0.bn2.bias - torch.Size([512]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer4.0.downsample.0.weight - torch.Size([512, 256, 1, 1]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer4.0.downsample.1.weight - torch.Size([512]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer4.0.downsample.1.bias - torch.Size([512]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer4.1.conv1.weight - torch.Size([512, 512, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer4.1.bn1.weight - torch.Size([512]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer4.1.bn1.bias - torch.Size([512]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.backbone.layer4.1.conv2.weight - torch.Size([512, 512, 3, 3]): KaimingInit: a=0, mode=fan_out, nonlinearity=relu, distribution =normal, bias=0

architecture.model.backbone.layer4.1.bn2.weight - torch.Size([512]): Initialized by user-defined init_weights in ResNet

architecture.model.backbone.layer4.1.bn2.bias - torch.Size([512]): The value is the same before and after calling init_weights of GeneralDistill

architecture.model.head.fc.weight - torch.Size([2, 512]): NormalInit: mean=0, std=0.01, bias=0

architecture.model.head.fc.bias - torch.Size([2]): NormalInit: mean=0, std=0.01, bias=0

distiller.teacher.backbone.conv1.weight - torch.Size([64, 3, 7, 7]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.bn1.weight - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.bn1.bias - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.0.conv1.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.0.bn1.weight - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.0.bn1.bias - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.0.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.0.bn2.weight - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.0.bn2.bias - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.1.conv1.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.1.bn1.weight - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.1.bn1.bias - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.1.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.1.bn2.weight - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.1.bn2.bias - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.2.conv1.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.2.bn1.weight - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.2.bn1.bias - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.2.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.2.bn2.weight - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer1.2.bn2.bias - torch.Size([64]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.conv1.weight - torch.Size([128, 64, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.bn1.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.bn1.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.bn2.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.bn2.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.downsample.0.weight - torch.Size([128, 64, 1, 1]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.downsample.1.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.0.downsample.1.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.1.conv1.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.1.bn1.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.1.bn1.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.1.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.1.bn2.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.1.bn2.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.2.conv1.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.2.bn1.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.2.bn1.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.2.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.2.bn2.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.2.bn2.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.3.conv1.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.3.bn1.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.3.bn1.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.3.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.3.bn2.weight - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer2.3.bn2.bias - torch.Size([128]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.conv1.weight - torch.Size([256, 128, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.bn1.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.bn1.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.bn2.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.bn2.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.downsample.0.weight - torch.Size([256, 128, 1, 1]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.downsample.1.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.0.downsample.1.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.1.conv1.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.1.bn1.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.1.bn1.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.1.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.1.bn2.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.1.bn2.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.2.conv1.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.2.bn1.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.2.bn1.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.2.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.2.bn2.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.2.bn2.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.3.conv1.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.3.bn1.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.3.bn1.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.3.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.3.bn2.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.3.bn2.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.4.conv1.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.4.bn1.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.4.bn1.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.4.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.4.bn2.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.4.bn2.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.5.conv1.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.5.bn1.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.5.bn1.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.5.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.5.bn2.weight - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer3.5.bn2.bias - torch.Size([256]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.conv1.weight - torch.Size([512, 256, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.bn1.weight - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.bn1.bias - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.bn2.weight - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.bn2.bias - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.downsample.0.weight - torch.Size([512, 256, 1, 1]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.downsample.1.weight - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.0.downsample.1.bias - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.1.conv1.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.1.bn1.weight - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.1.bn1.bias - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.1.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.1.bn2.weight - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.1.bn2.bias - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.2.conv1.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.2.bn1.weight - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.2.bn1.bias - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.2.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.2.bn2.weight - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.backbone.layer4.2.bn2.bias - torch.Size([512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.head.fc.weight - torch.Size([2, 512]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth

distiller.teacher.head.fc.bias - torch.Size([2]): PretrainedInit: load from /data/dataset_81/jd_result/epoch_100.pth 2022-05-06 14:08:26,739 - mmcls - INFO - Start running, host: root@iaep, work_dir: /data/dataset_81/jd_result/distill_mmcls 2022-05-06 14:08:26,740 - mmcls - INFO - Hooks will be executed in the following order: before_run: (VERY_HIGH ) StepLrUpdaterHook
(NORMAL ) CheckpointHook
(LOW ) EvalHook
(VERY_LOW ) TextLoggerHook

---

before_train_epoch: (VERY_HIGH ) StepLrUpdaterHook
(LOW ) IterTimerHook
(LOW ) EvalHook
(VERY_LOW ) TextLoggerHook

---

before_train_iter: (VERY_HIGH ) StepLrUpdaterHook
(LOW ) IterTimerHook
(LOW ) EvalHook

---

after_train_iter: (ABOVE_NORMAL) OptimizerHook
(NORMAL ) CheckpointHook
(LOW ) IterTimerHook
(LOW ) EvalHook
(VERY_LOW ) TextLoggerHook

---

after_train_epoch: (NORMAL ) CheckpointHook
(LOW ) EvalHook
(VERY_LOW ) TextLoggerHook

---

before_val_epoch: (LOW ) IterTimerHook
(VERY_LOW ) TextLoggerHook

---

before_val_iter: (LOW ) IterTimerHook

---

after_val_iter: (LOW ) IterTimerHook

---

after_val_epoch: (VERY_LOW ) TextLoggerHook

---

after_run: (VERY_LOW ) TextLoggerHook

---

2022-05-06 14:08:26,740 - mmcls - INFO - workflow: [('train', 1)], max: 100 epochs 2022-05-06 14:08:26,740 - mmcls - INFO - Checkpoints will be saved to /data/dataset_81/jd_result/distill_mmcls by HardDiskBackend. 2022-05-06 14:08:37,754 - mmcls - INFO - Epoch [1][100/201] lr: 1.000e-01, eta: 0:36:34, time: 0.110, data_time: 0.049, memory: 987, student.loss: 0.9828, distiller.distance_wise_loss.0: 1.1371, distiller.angle_wise_loss.0: 2.2090, loss: 4.3289 2022-05-06 14:08:45,535 - mmcls - INFO - Epoch [1][200/201] lr: 1.000e-01, eta: 0:31:05, time: 0.078, data_time: 0.028, memory: 987, student.loss: 0.6619, distiller.distance_wise_loss.0: 1.0240, distiller.angle_wise_loss.0: 1.8937, loss: 3.5797 2022-05-06 14:08:45,652 - mmcls - INFO - Saving checkpoint at 1 epochs 2022-05-06 14:08:52,250 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:08:52,251 - mmcls - INFO - Epoch(val) [1][51] accuracy_top-1: 63.8351 2022-05-06 14:09:01,200 - mmcls - INFO - Epoch [2][100/201] lr: 1.000e-01, eta: 0:30:19, time: 0.089, data_time: 0.040, memory: 987, student.loss: 0.6718, distiller.distance_wise_loss.0: 1.0049, distiller.angle_wise_loss.0: 1.8476, loss: 3.5244 2022-05-06 14:09:08,072 - mmcls - INFO - Epoch [2][200/201] lr: 1.000e-01, eta: 0:28:16, time: 0.069, data_time: 0.020, memory: 987, student.loss: 0.6273, distiller.distance_wise_loss.0: 1.0002, distiller.angle_wise_loss.0: 1.8209, loss: 3.4484 2022-05-06 14:09:08,169 - mmcls - INFO - Saving checkpoint at 2 epochs 2022-05-06 14:09:13,696 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:09:13,696 - mmcls - INFO - Epoch(val) [2][51] accuracy_top-1: 64.9594 2022-05-06 14:09:22,471 - mmcls - INFO - Epoch [3][100/201] lr: 1.000e-01, eta: 0:28:08, time: 0.087, data_time: 0.039, memory: 987, student.loss: 0.6213, distiller.distance_wise_loss.0: 0.9759, distiller.angle_wise_loss.0: 1.7838, loss: 3.3810 2022-05-06 14:09:29,571 - mmcls - INFO - Epoch [3][200/201] lr: 1.000e-01, eta: 0:27:10, time: 0.071, data_time: 0.021, memory: 987, student.loss: 0.6225, distiller.distance_wise_loss.0: 0.9646, distiller.angle_wise_loss.0: 1.7614, loss: 3.3486 2022-05-06 14:09:29,654 - mmcls - INFO - Saving checkpoint at 3 epochs 2022-05-06 14:09:35,280 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:09:35,281 - mmcls - INFO - Epoch(val) [3][51] accuracy_top-1: 73.8289 2022-05-06 14:09:44,301 - mmcls - INFO - Epoch [4][100/201] lr: 1.000e-01, eta: 0:27:16, time: 0.090, data_time: 0.042, memory: 987, student.loss: 0.6073, distiller.distance_wise_loss.0: 0.9171, distiller.angle_wise_loss.0: 1.6967, loss: 3.2211 2022-05-06 14:09:51,386 - mmcls - INFO - Epoch [4][200/201] lr: 1.000e-01, eta: 0:26:35, time: 0.071, data_time: 0.023, memory: 987, student.loss: 0.5900, distiller.distance_wise_loss.0: 0.9492, distiller.angle_wise_loss.0: 1.7251, loss: 3.2643 2022-05-06 14:09:51,486 - mmcls - INFO - Saving checkpoint at 4 epochs 2022-05-06 14:09:56,894 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:09:56,895 - mmcls - INFO - Epoch(val) [4][51] accuracy_top-1: 67.5203 2022-05-06 14:10:06,230 - mmcls - INFO - Epoch [5][100/201] lr: 1.000e-01, eta: 0:26:46, time: 0.093, data_time: 0.043, memory: 987, student.loss: 0.5823, distiller.distance_wise_loss.0: 0.8944, distiller.angle_wise_loss.0: 1.6658, loss: 3.1425 2022-05-06 14:10:13,585 - mmcls - INFO - Epoch [5][200/201] lr: 1.000e-01, eta: 0:26:19, time: 0.074, data_time: 0.026, memory: 987, student.loss: 0.5585, distiller.distance_wise_loss.0: 0.9061, distiller.angle_wise_loss.0: 1.6760, loss: 3.1407 2022-05-06 14:10:14,403 - mmcls - INFO - Saving checkpoint at 5 epochs 2022-05-06 14:10:19,707 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:10:19,708 - mmcls - INFO - Epoch(val) [5][51] accuracy_top-1: 64.3973 2022-05-06 14:10:28,438 - mmcls - INFO - Epoch [6][100/201] lr: 1.000e-01, eta: 0:26:16, time: 0.087, data_time: 0.039, memory: 987, student.loss: 0.5537, distiller.distance_wise_loss.0: 0.8952, distiller.angle_wise_loss.0: 1.6374, loss: 3.0862 2022-05-06 14:10:38,285 - mmcls - INFO - Epoch [6][200/201] lr: 1.000e-01, eta: 0:26:32, time: 0.098, data_time: 0.050, memory: 987, student.loss: 0.5356, distiller.distance_wise_loss.0: 0.8301, distiller.angle_wise_loss.0: 1.5311, loss: 2.8968 2022-05-06 14:10:38,383 - mmcls - INFO - Saving checkpoint at 6 epochs 2022-05-06 14:10:43,719 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:10:43,719 - mmcls - INFO - Epoch(val) [6][51] accuracy_top-1: 71.7052 2022-05-06 14:10:53,019 - mmcls - INFO - Epoch [7][100/201] lr: 1.000e-01, eta: 0:26:34, time: 0.092, data_time: 0.040, memory: 987, student.loss: 0.5157, distiller.distance_wise_loss.0: 0.8041, distiller.angle_wise_loss.0: 1.5128, loss: 2.8326 2022-05-06 14:11:00,093 - mmcls - INFO - Epoch [7][200/201] lr: 1.000e-01, eta: 0:26:07, time: 0.071, data_time: 0.022, memory: 987, student.loss: 0.4989, distiller.distance_wise_loss.0: 0.7740, distiller.angle_wise_loss.0: 1.4521, loss: 2.7251 2022-05-06 14:11:00,264 - mmcls - INFO - Saving checkpoint at 7 epochs 2022-05-06 14:11:05,745 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:11:05,746 - mmcls - INFO - Epoch(val) [7][51] accuracy_top-1: 81.0743 2022-05-06 14:11:14,996 - mmcls - INFO - Epoch [8][100/201] lr: 1.000e-01, eta: 0:26:07, time: 0.092, data_time: 0.042, memory: 987, student.loss: 0.4960, distiller.distance_wise_loss.0: 0.8011, distiller.angle_wise_loss.0: 1.4928, loss: 2.7899 2022-05-06 14:11:21,553 - mmcls - INFO - Epoch [8][200/201] lr: 1.000e-01, eta: 0:25:37, time: 0.066, data_time: 0.016, memory: 987, student.loss: 0.4737, distiller.distance_wise_loss.0: 0.7279, distiller.angle_wise_loss.0: 1.4027, loss: 2.6043 2022-05-06 14:11:21,918 - mmcls - INFO - Saving checkpoint at 8 epochs 2022-05-06 14:11:27,798 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:11:27,799 - mmcls - INFO - Epoch(val) [8][51] accuracy_top-1: 79.6377 2022-05-06 14:11:37,137 - mmcls - INFO - Epoch [9][100/201] lr: 1.000e-01, eta: 0:25:38, time: 0.093, data_time: 0.044, memory: 987, student.loss: 0.4592, distiller.distance_wise_loss.0: 0.7025, distiller.angle_wise_loss.0: 1.3282, loss: 2.4898 2022-05-06 14:11:43,887 - mmcls - INFO - Epoch [9][200/201] lr: 1.000e-01, eta: 0:25:13, time: 0.067, data_time: 0.018, memory: 987, student.loss: 0.4482, distiller.distance_wise_loss.0: 0.6485, distiller.angle_wise_loss.0: 1.2380, loss: 2.3346 2022-05-06 14:11:43,983 - mmcls - INFO - Saving checkpoint at 9 epochs 2022-05-06 14:11:49,442 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:11:49,442 - mmcls - INFO - Epoch(val) [9][51] accuracy_top-1: 76.1399 2022-05-06 14:11:58,549 - mmcls - INFO - Epoch [10][100/201] lr: 1.000e-01, eta: 0:25:12, time: 0.091, data_time: 0.041, memory: 987, student.loss: 0.4349, distiller.distance_wise_loss.0: 0.6269, distiller.angle_wise_loss.0: 1.2138, loss: 2.2757 2022-05-06 14:12:05,442 - mmcls - INFO - Epoch [10][200/201] lr: 1.000e-01, eta: 0:24:51, time: 0.069, data_time: 0.019, memory: 987, student.loss: 0.4503, distiller.distance_wise_loss.0: 0.6299, distiller.angle_wise_loss.0: 1.2086, loss: 2.2888 2022-05-06 14:12:05,526 - mmcls - INFO - Saving checkpoint at 10 epochs 2022-05-06 14:12:12,406 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:12:12,407 - mmcls - INFO - Epoch(val) [10][51] accuracy_top-1: 79.9500 2022-05-06 14:12:21,682 - mmcls - INFO - Epoch [11][100/201] lr: 1.000e-01, eta: 0:24:46, time: 0.088, data_time: 0.038, memory: 987, student.loss: 0.4131, distiller.distance_wise_loss.0: 0.5646, distiller.angle_wise_loss.0: 1.0934, loss: 2.0711 2022-05-06 14:12:28,532 - mmcls - INFO - Epoch [11][200/201] lr: 1.000e-01, eta: 0:24:26, time: 0.068, data_time: 0.018, memory: 987, student.loss: 0.3962, distiller.distance_wise_loss.0: 0.5447, distiller.angle_wise_loss.0: 1.0623, loss: 2.0031 2022-05-06 14:12:29,643 - mmcls - INFO - Saving checkpoint at 11 epochs 2022-05-06 14:12:35,442 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:12:35,443 - mmcls - INFO - Epoch(val) [11][51] accuracy_top-1: 76.1399 2022-05-06 14:12:44,580 - mmcls - INFO - Epoch [12][100/201] lr: 1.000e-01, eta: 0:24:24, time: 0.091, data_time: 0.042, memory: 987, student.loss: 0.3838, distiller.distance_wise_loss.0: 0.4959, distiller.angle_wise_loss.0: 0.9917, loss: 1.8715 2022-05-06 14:12:51,393 - mmcls - INFO - Epoch [12][200/201] lr: 1.000e-01, eta: 0:24:06, time: 0.068, data_time: 0.020, memory: 987, student.loss: 0.3633, distiller.distance_wise_loss.0: 0.4862, distiller.angle_wise_loss.0: 0.9538, loss: 1.8034 2022-05-06 14:12:51,483 - mmcls - INFO - Saving checkpoint at 12 epochs 2022-05-06 14:12:57,967 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:12:57,967 - mmcls - INFO - Epoch(val) [12][51] accuracy_top-1: 83.7602 2022-05-06 14:13:07,131 - mmcls - INFO - Epoch [13][100/201] lr: 1.000e-01, eta: 0:24:03, time: 0.091, data_time: 0.043, memory: 987, student.loss: 0.3767, distiller.distance_wise_loss.0: 0.4624, distiller.angle_wise_loss.0: 0.9215, loss: 1.7606 2022-05-06 14:13:14,190 - mmcls - INFO - Epoch [13][200/201] lr: 1.000e-01, eta: 0:23:47, time: 0.071, data_time: 0.020, memory: 987, student.loss: 0.3318, distiller.distance_wise_loss.0: 0.3996, distiller.angle_wise_loss.0: 0.8175, loss: 1.5489 2022-05-06 14:13:14,700 - mmcls - INFO - Saving checkpoint at 13 epochs 2022-05-06 14:13:20,016 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:13:20,016 - mmcls - INFO - Epoch(val) [13][51] accuracy_top-1: 90.0062 2022-05-06 14:13:29,287 - mmcls - INFO - Epoch [14][100/201] lr: 1.000e-01, eta: 0:23:46, time: 0.092, data_time: 0.043, memory: 987, student.loss: 0.3164, distiller.distance_wise_loss.0: 0.3747, distiller.angle_wise_loss.0: 0.7694, loss: 1.4605 2022-05-06 14:13:36,154 - mmcls - INFO - Epoch [14][200/201] lr: 1.000e-01, eta: 0:23:29, time: 0.069, data_time: 0.020, memory: 987, student.loss: 0.2993, distiller.distance_wise_loss.0: 0.3561, distiller.angle_wise_loss.0: 0.7309, loss: 1.3863 2022-05-06 14:13:36,995 - mmcls - INFO - Saving checkpoint at 14 epochs 2022-05-06 14:13:42,433 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:13:42,434 - mmcls - INFO - Epoch(val) [14][51] accuracy_top-1: 90.6933 2022-05-06 14:13:51,526 - mmcls - INFO - Epoch [15][100/201] lr: 1.000e-01, eta: 0:23:25, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.2992, distiller.distance_wise_loss.0: 0.3280, distiller.angle_wise_loss.0: 0.6940, loss: 1.3212 2022-05-06 14:13:58,583 - mmcls - INFO - Epoch [15][200/201] lr: 1.000e-01, eta: 0:23:11, time: 0.071, data_time: 0.021, memory: 987, student.loss: 0.3126, distiller.distance_wise_loss.0: 0.3386, distiller.angle_wise_loss.0: 0.7022, loss: 1.3534 2022-05-06 14:13:58,676 - mmcls - INFO - Saving checkpoint at 15 epochs 2022-05-06 14:14:05,591 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:14:05,592 - mmcls - INFO - Epoch(val) [15][51] accuracy_top-1: 84.9469 2022-05-06 14:14:15,191 - mmcls - INFO - Epoch [16][100/201] lr: 1.000e-01, eta: 0:23:10, time: 0.096, data_time: 0.036, memory: 987, student.loss: 0.2873, distiller.distance_wise_loss.0: 0.3055, distiller.angle_wise_loss.0: 0.6454, loss: 1.2381 2022-05-06 14:14:22,399 - mmcls - INFO - Epoch [16][200/201] lr: 1.000e-01, eta: 0:22:56, time: 0.072, data_time: 0.026, memory: 987, student.loss: 0.2853, distiller.distance_wise_loss.0: 0.2995, distiller.angle_wise_loss.0: 0.6389, loss: 1.2237 2022-05-06 14:14:22,472 - mmcls - INFO - Saving checkpoint at 16 epochs 2022-05-06 14:14:27,840 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:14:27,840 - mmcls - INFO - Epoch(val) [16][51] accuracy_top-1: 91.2555 2022-05-06 14:14:36,805 - mmcls - INFO - Epoch [17][100/201] lr: 1.000e-01, eta: 0:22:52, time: 0.089, data_time: 0.041, memory: 987, student.loss: 0.2891, distiller.distance_wise_loss.0: 0.3055, distiller.angle_wise_loss.0: 0.6416, loss: 1.2361 2022-05-06 14:14:43,543 - mmcls - INFO - Epoch [17][200/201] lr: 1.000e-01, eta: 0:22:36, time: 0.067, data_time: 0.019, memory: 987, student.loss: 0.2875, distiller.distance_wise_loss.0: 0.2949, distiller.angle_wise_loss.0: 0.6181, loss: 1.2005 2022-05-06 14:14:45,013 - mmcls - INFO - Saving checkpoint at 17 epochs 2022-05-06 14:14:50,593 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:14:50,593 - mmcls - INFO - Epoch(val) [17][51] accuracy_top-1: 91.9425 2022-05-06 14:14:59,729 - mmcls - INFO - Epoch [18][100/201] lr: 1.000e-01, eta: 0:22:32, time: 0.091, data_time: 0.042, memory: 987, student.loss: 0.2618, distiller.distance_wise_loss.0: 0.2382, distiller.angle_wise_loss.0: 0.5261, loss: 1.0260 2022-05-06 14:15:06,632 - mmcls - INFO - Epoch [18][200/201] lr: 1.000e-01, eta: 0:22:18, time: 0.069, data_time: 0.020, memory: 987, student.loss: 0.2628, distiller.distance_wise_loss.0: 0.2263, distiller.angle_wise_loss.0: 0.5000, loss: 0.9891 2022-05-06 14:15:07,322 - mmcls - INFO - Saving checkpoint at 18 epochs 2022-05-06 14:15:12,662 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:15:12,663 - mmcls - INFO - Epoch(val) [18][51] accuracy_top-1: 93.6914 2022-05-06 14:15:22,436 - mmcls - INFO - Epoch [19][100/201] lr: 1.000e-01, eta: 0:22:15, time: 0.093, data_time: 0.044, memory: 987, student.loss: 0.2562, distiller.distance_wise_loss.0: 0.2306, distiller.angle_wise_loss.0: 0.5049, loss: 0.9918 2022-05-06 14:15:28,712 - mmcls - INFO - Epoch [19][200/201] lr: 1.000e-01, eta: 0:21:59, time: 0.063, data_time: 0.016, memory: 987, student.loss: 0.2610, distiller.distance_wise_loss.0: 0.2296, distiller.angle_wise_loss.0: 0.5020, loss: 0.9926 2022-05-06 14:15:28,805 - mmcls - INFO - Saving checkpoint at 19 epochs 2022-05-06 14:15:34,077 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:15:34,078 - mmcls - INFO - Epoch(val) [19][51] accuracy_top-1: 89.8813 2022-05-06 14:15:43,167 - mmcls - INFO - Epoch [20][100/201] lr: 1.000e-01, eta: 0:21:54, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.2429, distiller.distance_wise_loss.0: 0.2139, distiller.angle_wise_loss.0: 0.4662, loss: 0.9230 2022-05-06 14:15:50,300 - mmcls - INFO - Epoch [20][200/201] lr: 1.000e-01, eta: 0:21:42, time: 0.071, data_time: 0.020, memory: 987, student.loss: 0.2474, distiller.distance_wise_loss.0: 0.2241, distiller.angle_wise_loss.0: 0.4871, loss: 0.9586 2022-05-06 14:15:50,412 - mmcls - INFO - Saving checkpoint at 20 epochs 2022-05-06 14:15:55,741 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:15:55,741 - mmcls - INFO - Epoch(val) [20][51] accuracy_top-1: 91.9425 2022-05-06 14:16:04,953 - mmcls - INFO - Epoch [21][100/201] lr: 1.000e-01, eta: 0:21:37, time: 0.089, data_time: 0.041, memory: 987, student.loss: 0.2418, distiller.distance_wise_loss.0: 0.2207, distiller.angle_wise_loss.0: 0.4888, loss: 0.9512 2022-05-06 14:16:12,684 - mmcls - INFO - Epoch [21][200/201] lr: 1.000e-01, eta: 0:21:27, time: 0.077, data_time: 0.014, memory: 987, student.loss: 0.2291, distiller.distance_wise_loss.0: 0.1941, distiller.angle_wise_loss.0: 0.4334, loss: 0.8566 2022-05-06 14:16:12,808 - mmcls - INFO - Saving checkpoint at 21 epochs 2022-05-06 14:16:18,435 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:16:18,436 - mmcls - INFO - Epoch(val) [21][51] accuracy_top-1: 93.8788 2022-05-06 14:16:28,245 - mmcls - INFO - Epoch [22][100/201] lr: 1.000e-01, eta: 0:21:22, time: 0.091, data_time: 0.043, memory: 987, student.loss: 0.2586, distiller.distance_wise_loss.0: 0.2121, distiller.angle_wise_loss.0: 0.4681, loss: 0.9388 2022-05-06 14:16:36,008 - mmcls - INFO - Epoch [22][200/201] lr: 1.000e-01, eta: 0:21:13, time: 0.078, data_time: 0.028, memory: 987, student.loss: 0.2232, distiller.distance_wise_loss.0: 0.1983, distiller.angle_wise_loss.0: 0.4411, loss: 0.8627 2022-05-06 14:16:39,406 - mmcls - INFO - Saving checkpoint at 22 epochs 2022-05-06 14:16:46,052 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:16:46,053 - mmcls - INFO - Epoch(val) [22][51] accuracy_top-1: 93.5041 2022-05-06 14:16:55,248 - mmcls - INFO - Epoch [23][100/201] lr: 1.000e-01, eta: 0:21:08, time: 0.092, data_time: 0.044, memory: 987, student.loss: 0.2299, distiller.distance_wise_loss.0: 0.1775, distiller.angle_wise_loss.0: 0.4017, loss: 0.8091 2022-05-06 14:17:02,579 - mmcls - INFO - Epoch [23][200/201] lr: 1.000e-01, eta: 0:20:57, time: 0.073, data_time: 0.023, memory: 987, student.loss: 0.2243, distiller.distance_wise_loss.0: 0.1889, distiller.angle_wise_loss.0: 0.4177, loss: 0.8309 2022-05-06 14:17:03,163 - mmcls - INFO - Saving checkpoint at 23 epochs 2022-05-06 14:17:12,369 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:17:12,370 - mmcls - INFO - Epoch(val) [23][51] accuracy_top-1: 93.8788 2022-05-06 14:17:21,622 - mmcls - INFO - Epoch [24][100/201] lr: 1.000e-01, eta: 0:20:52, time: 0.092, data_time: 0.041, memory: 987, student.loss: 0.2277, distiller.distance_wise_loss.0: 0.1796, distiller.angle_wise_loss.0: 0.4096, loss: 0.8170 2022-05-06 14:17:28,859 - mmcls - INFO - Epoch [24][200/201] lr: 1.000e-01, eta: 0:20:41, time: 0.072, data_time: 0.022, memory: 987, student.loss: 0.2454, distiller.distance_wise_loss.0: 0.1918, distiller.angle_wise_loss.0: 0.4268, loss: 0.8641 2022-05-06 14:17:28,953 - mmcls - INFO - Saving checkpoint at 24 epochs 2022-05-06 14:17:34,882 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:17:34,882 - mmcls - INFO - Epoch(val) [24][51] accuracy_top-1: 93.6290 2022-05-06 14:17:45,326 - mmcls - INFO - Epoch [25][100/201] lr: 1.000e-01, eta: 0:20:35, time: 0.090, data_time: 0.042, memory: 987, student.loss: 0.2109, distiller.distance_wise_loss.0: 0.1571, distiller.angle_wise_loss.0: 0.3615, loss: 0.7296 2022-05-06 14:17:52,244 - mmcls - INFO - Epoch [25][200/201] lr: 1.000e-01, eta: 0:20:24, time: 0.069, data_time: 0.018, memory: 987, student.loss: 0.2278, distiller.distance_wise_loss.0: 0.1712, distiller.angle_wise_loss.0: 0.3819, loss: 0.7809 2022-05-06 14:17:52,372 - mmcls - INFO - Saving checkpoint at 25 epochs 2022-05-06 14:17:58,771 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:17:58,772 - mmcls - INFO - Epoch(val) [25][51] accuracy_top-1: 93.0668 2022-05-06 14:18:08,143 - mmcls - INFO - Epoch [26][100/201] lr: 1.000e-01, eta: 0:20:18, time: 0.091, data_time: 0.042, memory: 987, student.loss: 0.2233, distiller.distance_wise_loss.0: 0.1777, distiller.angle_wise_loss.0: 0.4042, loss: 0.8051 2022-05-06 14:18:15,824 - mmcls - INFO - Epoch [26][200/201] lr: 1.000e-01, eta: 0:20:09, time: 0.077, data_time: 0.029, memory: 987, student.loss: 0.2207, distiller.distance_wise_loss.0: 0.1739, distiller.angle_wise_loss.0: 0.3921, loss: 0.7867 2022-05-06 14:18:16,081 - mmcls - INFO - Saving checkpoint at 26 epochs 2022-05-06 14:18:21,383 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:18:21,383 - mmcls - INFO - Epoch(val) [26][51] accuracy_top-1: 94.1911 2022-05-06 14:18:30,267 - mmcls - INFO - Epoch [27][100/201] lr: 1.000e-01, eta: 0:20:02, time: 0.088, data_time: 0.034, memory: 987, student.loss: 0.2071, distiller.distance_wise_loss.0: 0.1516, distiller.angle_wise_loss.0: 0.3514, loss: 0.7100 2022-05-06 14:18:37,213 - mmcls - INFO - Epoch [27][200/201] lr: 1.000e-01, eta: 0:19:51, time: 0.069, data_time: 0.018, memory: 987, student.loss: 0.2061, distiller.distance_wise_loss.0: 0.1576, distiller.angle_wise_loss.0: 0.3627, loss: 0.7264 2022-05-06 14:18:37,310 - mmcls - INFO - Saving checkpoint at 27 epochs 2022-05-06 14:18:43,959 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:18:43,959 - mmcls - INFO - Epoch(val) [27][51] accuracy_top-1: 87.8826 2022-05-06 14:18:53,045 - mmcls - INFO - Epoch [28][100/201] lr: 1.000e-01, eta: 0:19:44, time: 0.089, data_time: 0.041, memory: 987, student.loss: 0.2017, distiller.distance_wise_loss.0: 0.1606, distiller.angle_wise_loss.0: 0.3662, loss: 0.7285 2022-05-06 14:19:05,493 - mmcls - INFO - Epoch [28][200/201] lr: 1.000e-01, eta: 0:19:47, time: 0.124, data_time: 0.075, memory: 987, student.loss: 0.2239, distiller.distance_wise_loss.0: 0.1583, distiller.angle_wise_loss.0: 0.3566, loss: 0.7388 2022-05-06 14:19:06,039 - mmcls - INFO - Saving checkpoint at 28 epochs 2022-05-06 14:19:12,133 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:19:12,133 - mmcls - INFO - Epoch(val) [28][51] accuracy_top-1: 95.1280 2022-05-06 14:19:22,137 - mmcls - INFO - Epoch [29][100/201] lr: 1.000e-01, eta: 0:19:41, time: 0.089, data_time: 0.040, memory: 987, student.loss: 0.2141, distiller.distance_wise_loss.0: 0.1566, distiller.angle_wise_loss.0: 0.3619, loss: 0.7326 2022-05-06 14:19:29,394 - mmcls - INFO - Epoch [29][200/201] lr: 1.000e-01, eta: 0:19:30, time: 0.073, data_time: 0.023, memory: 987, student.loss: 0.2110, distiller.distance_wise_loss.0: 0.1463, distiller.angle_wise_loss.0: 0.3419, loss: 0.6991 2022-05-06 14:19:29,481 - mmcls - INFO - Saving checkpoint at 29 epochs 2022-05-06 14:19:35,831 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:19:35,831 - mmcls - INFO - Epoch(val) [29][51] accuracy_top-1: 95.2530 2022-05-06 14:19:44,981 - mmcls - INFO - Epoch [30][100/201] lr: 1.000e-01, eta: 0:19:24, time: 0.091, data_time: 0.041, memory: 987, student.loss: 0.2043, distiller.distance_wise_loss.0: 0.1468, distiller.angle_wise_loss.0: 0.3362, loss: 0.6872 2022-05-06 14:19:54,048 - mmcls - INFO - Epoch [30][200/201] lr: 1.000e-01, eta: 0:19:17, time: 0.091, data_time: 0.044, memory: 987, student.loss: 0.2105, distiller.distance_wise_loss.0: 0.1468, distiller.angle_wise_loss.0: 0.3422, loss: 0.6994 2022-05-06 14:19:54,474 - mmcls - INFO - Saving checkpoint at 30 epochs 2022-05-06 14:20:00,289 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:20:00,290 - mmcls - INFO - Epoch(val) [30][51] accuracy_top-1: 94.4410 2022-05-06 14:20:10,123 - mmcls - INFO - Epoch [31][100/201] lr: 1.000e-02, eta: 0:19:12, time: 0.097, data_time: 0.036, memory: 987, student.loss: 0.1700, distiller.distance_wise_loss.0: 0.1184, distiller.angle_wise_loss.0: 0.2872, loss: 0.5757 2022-05-06 14:20:16,763 - mmcls - INFO - Epoch [31][200/201] lr: 1.000e-02, eta: 0:19:00, time: 0.066, data_time: 0.019, memory: 987, student.loss: 0.1725, distiller.distance_wise_loss.0: 0.0995, distiller.angle_wise_loss.0: 0.2417, loss: 0.5137 2022-05-06 14:20:16,858 - mmcls - INFO - Saving checkpoint at 31 epochs 2022-05-06 14:20:22,515 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:20:22,515 - mmcls - INFO - Epoch(val) [31][51] accuracy_top-1: 95.3154 2022-05-06 14:20:31,670 - mmcls - INFO - Epoch [32][100/201] lr: 1.000e-02, eta: 0:18:54, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.1626, distiller.distance_wise_loss.0: 0.0972, distiller.angle_wise_loss.0: 0.2333, loss: 0.4932 2022-05-06 14:20:38,967 - mmcls - INFO - Epoch [32][200/201] lr: 1.000e-02, eta: 0:18:43, time: 0.073, data_time: 0.023, memory: 987, student.loss: 0.1792, distiller.distance_wise_loss.0: 0.0922, distiller.angle_wise_loss.0: 0.2214, loss: 0.4928 2022-05-06 14:20:39,056 - mmcls - INFO - Saving checkpoint at 32 epochs 2022-05-06 14:20:45,389 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:20:45,389 - mmcls - INFO - Epoch(val) [32][51] accuracy_top-1: 95.8776 2022-05-06 14:20:55,045 - mmcls - INFO - Epoch [33][100/201] lr: 1.000e-02, eta: 0:18:36, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.1675, distiller.distance_wise_loss.0: 0.0835, distiller.angle_wise_loss.0: 0.2115, loss: 0.4625 2022-05-06 14:21:01,564 - mmcls - INFO - Epoch [33][200/201] lr: 1.000e-02, eta: 0:18:25, time: 0.065, data_time: 0.015, memory: 987, student.loss: 0.1622, distiller.distance_wise_loss.0: 0.0893, distiller.angle_wise_loss.0: 0.2171, loss: 0.4686 2022-05-06 14:21:01,672 - mmcls - INFO - Saving checkpoint at 33 epochs 2022-05-06 14:21:08,493 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:21:08,493 - mmcls - INFO - Epoch(val) [33][51] accuracy_top-1: 95.9400 2022-05-06 14:21:18,014 - mmcls - INFO - Epoch [34][100/201] lr: 1.000e-02, eta: 0:18:18, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.1691, distiller.distance_wise_loss.0: 0.0860, distiller.angle_wise_loss.0: 0.2141, loss: 0.4693 2022-05-06 14:21:25,464 - mmcls - INFO - Epoch [34][200/201] lr: 1.000e-02, eta: 0:18:08, time: 0.074, data_time: 0.025, memory: 987, student.loss: 0.1591, distiller.distance_wise_loss.0: 0.0876, distiller.angle_wise_loss.0: 0.2120, loss: 0.4586 2022-05-06 14:21:25,556 - mmcls - INFO - Saving checkpoint at 34 epochs 2022-05-06 14:21:31,732 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:21:31,732 - mmcls - INFO - Epoch(val) [34][51] accuracy_top-1: 95.2530 2022-05-06 14:21:40,831 - mmcls - INFO - Epoch [35][100/201] lr: 1.000e-02, eta: 0:18:01, time: 0.090, data_time: 0.039, memory: 987, student.loss: 0.1665, distiller.distance_wise_loss.0: 0.0855, distiller.angle_wise_loss.0: 0.2078, loss: 0.4597 2022-05-06 14:21:47,710 - mmcls - INFO - Epoch [35][200/201] lr: 1.000e-02, eta: 0:17:51, time: 0.069, data_time: 0.018, memory: 987, student.loss: 0.1629, distiller.distance_wise_loss.0: 0.0788, distiller.angle_wise_loss.0: 0.1949, loss: 0.4366 2022-05-06 14:21:47,810 - mmcls - INFO - Saving checkpoint at 35 epochs 2022-05-06 14:21:55,261 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:21:55,262 - mmcls - INFO - Epoch(val) [35][51] accuracy_top-1: 95.8776 2022-05-06 14:22:04,355 - mmcls - INFO - Epoch [36][100/201] lr: 1.000e-02, eta: 0:17:44, time: 0.091, data_time: 0.040, memory: 987, student.loss: 0.1659, distiller.distance_wise_loss.0: 0.0824, distiller.angle_wise_loss.0: 0.1993, loss: 0.4476 2022-05-06 14:22:11,213 - mmcls - INFO - Epoch [36][200/201] lr: 1.000e-02, eta: 0:17:33, time: 0.069, data_time: 0.019, memory: 987, student.loss: 0.1680, distiller.distance_wise_loss.0: 0.0796, distiller.angle_wise_loss.0: 0.1954, loss: 0.4430 2022-05-06 14:22:12,000 - mmcls - INFO - Saving checkpoint at 36 epochs 2022-05-06 14:22:17,243 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:22:17,243 - mmcls - INFO - Epoch(val) [36][51] accuracy_top-1: 95.5028 2022-05-06 14:22:27,117 - mmcls - INFO - Epoch [37][100/201] lr: 1.000e-02, eta: 0:17:27, time: 0.098, data_time: 0.042, memory: 987, student.loss: 0.1568, distiller.distance_wise_loss.0: 0.0825, distiller.angle_wise_loss.0: 0.2044, loss: 0.4437 2022-05-06 14:22:33,647 - mmcls - INFO - Epoch [37][200/201] lr: 1.000e-02, eta: 0:17:16, time: 0.065, data_time: 0.012, memory: 987, student.loss: 0.1662, distiller.distance_wise_loss.0: 0.0814, distiller.angle_wise_loss.0: 0.1975, loss: 0.4450 2022-05-06 14:22:33,737 - mmcls - INFO - Saving checkpoint at 37 epochs 2022-05-06 14:22:39,806 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:22:39,806 - mmcls - INFO - Epoch(val) [37][51] accuracy_top-1: 96.1899 2022-05-06 14:22:49,025 - mmcls - INFO - Epoch [38][100/201] lr: 1.000e-02, eta: 0:17:10, time: 0.092, data_time: 0.043, memory: 987, student.loss: 0.1612, distiller.distance_wise_loss.0: 0.0808, distiller.angle_wise_loss.0: 0.1992, loss: 0.4412 2022-05-06 14:22:56,218 - mmcls - INFO - Epoch [38][200/201] lr: 1.000e-02, eta: 0:17:00, time: 0.072, data_time: 0.024, memory: 987, student.loss: 0.1585, distiller.distance_wise_loss.0: 0.0757, distiller.angle_wise_loss.0: 0.1897, loss: 0.4239 2022-05-06 14:22:56,735 - mmcls - INFO - Saving checkpoint at 38 epochs 2022-05-06 14:23:03,296 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:23:03,296 - mmcls - INFO - Epoch(val) [38][51] accuracy_top-1: 95.7527 2022-05-06 14:23:13,277 - mmcls - INFO - Epoch [39][100/201] lr: 1.000e-02, eta: 0:16:53, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.1504, distiller.distance_wise_loss.0: 0.0812, distiller.angle_wise_loss.0: 0.1947, loss: 0.4263 2022-05-06 14:23:20,309 - mmcls - INFO - Epoch [39][200/201] lr: 1.000e-02, eta: 0:16:43, time: 0.070, data_time: 0.022, memory: 987, student.loss: 0.1498, distiller.distance_wise_loss.0: 0.0746, distiller.angle_wise_loss.0: 0.1868, loss: 0.4111 2022-05-06 14:23:20,403 - mmcls - INFO - Saving checkpoint at 39 epochs 2022-05-06 14:23:26,546 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:23:26,547 - mmcls - INFO - Epoch(val) [39][51] accuracy_top-1: 96.0025 2022-05-06 14:23:36,854 - mmcls - INFO - Epoch [40][100/201] lr: 1.000e-02, eta: 0:16:35, time: 0.086, data_time: 0.040, memory: 987, student.loss: 0.1579, distiller.distance_wise_loss.0: 0.0796, distiller.angle_wise_loss.0: 0.1915, loss: 0.4291 2022-05-06 14:23:43,818 - mmcls - INFO - Epoch [40][200/201] lr: 1.000e-02, eta: 0:16:25, time: 0.070, data_time: 0.022, memory: 987, student.loss: 0.1741, distiller.distance_wise_loss.0: 0.0761, distiller.angle_wise_loss.0: 0.1886, loss: 0.4388 2022-05-06 14:23:46,459 - mmcls - INFO - Saving checkpoint at 40 epochs 2022-05-06 14:23:54,134 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:23:54,135 - mmcls - INFO - Epoch(val) [40][51] accuracy_top-1: 96.1899 2022-05-06 14:24:03,896 - mmcls - INFO - Epoch [41][100/201] lr: 1.000e-02, eta: 0:16:18, time: 0.094, data_time: 0.044, memory: 987, student.loss: 0.1519, distiller.distance_wise_loss.0: 0.0822, distiller.angle_wise_loss.0: 0.1974, loss: 0.4316 2022-05-06 14:24:11,160 - mmcls - INFO - Epoch [41][200/201] lr: 1.000e-02, eta: 0:16:09, time: 0.073, data_time: 0.014, memory: 987, student.loss: 0.1568, distiller.distance_wise_loss.0: 0.0722, distiller.angle_wise_loss.0: 0.1804, loss: 0.4094 2022-05-06 14:24:11,250 - mmcls - INFO - Saving checkpoint at 41 epochs 2022-05-06 14:24:17,440 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:24:17,441 - mmcls - INFO - Epoch(val) [41][51] accuracy_top-1: 95.9400 2022-05-06 14:24:26,366 - mmcls - INFO - Epoch [42][100/201] lr: 1.000e-02, eta: 0:16:01, time: 0.086, data_time: 0.039, memory: 987, student.loss: 0.1568, distiller.distance_wise_loss.0: 0.0778, distiller.angle_wise_loss.0: 0.1901, loss: 0.4246 2022-05-06 14:24:33,496 - mmcls - INFO - Epoch [42][200/201] lr: 1.000e-02, eta: 0:15:51, time: 0.071, data_time: 0.023, memory: 987, student.loss: 0.1562, distiller.distance_wise_loss.0: 0.0763, distiller.angle_wise_loss.0: 0.1862, loss: 0.4186 2022-05-06 14:24:33,578 - mmcls - INFO - Saving checkpoint at 42 epochs 2022-05-06 14:24:39,699 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:24:39,699 - mmcls - INFO - Epoch(val) [42][51] accuracy_top-1: 95.6277 2022-05-06 14:24:48,575 - mmcls - INFO - Epoch [43][100/201] lr: 1.000e-02, eta: 0:15:44, time: 0.088, data_time: 0.039, memory: 987, student.loss: 0.1554, distiller.distance_wise_loss.0: 0.0730, distiller.angle_wise_loss.0: 0.1835, loss: 0.4119 2022-05-06 14:24:55,646 - mmcls - INFO - Epoch [43][200/201] lr: 1.000e-02, eta: 0:15:34, time: 0.071, data_time: 0.020, memory: 987, student.loss: 0.1523, distiller.distance_wise_loss.0: 0.0705, distiller.angle_wise_loss.0: 0.1755, loss: 0.3982 2022-05-06 14:24:56,399 - mmcls - INFO - Saving checkpoint at 43 epochs 2022-05-06 14:25:02,839 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:25:02,839 - mmcls - INFO - Epoch(val) [43][51] accuracy_top-1: 95.8151 2022-05-06 14:25:12,095 - mmcls - INFO - Epoch [44][100/201] lr: 1.000e-02, eta: 0:15:27, time: 0.092, data_time: 0.042, memory: 987, student.loss: 0.1663, distiller.distance_wise_loss.0: 0.0723, distiller.angle_wise_loss.0: 0.1795, loss: 0.4180 2022-05-06 14:25:19,599 - mmcls - INFO - Epoch [44][200/201] lr: 1.000e-02, eta: 0:15:18, time: 0.075, data_time: 0.016, memory: 987, student.loss: 0.1592, distiller.distance_wise_loss.0: 0.0723, distiller.angle_wise_loss.0: 0.1788, loss: 0.4104 2022-05-06 14:25:19,717 - mmcls - INFO - Saving checkpoint at 44 epochs 2022-05-06 14:25:26,173 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:25:26,174 - mmcls - INFO - Epoch(val) [44][51] accuracy_top-1: 95.6902 2022-05-06 14:25:35,908 - mmcls - INFO - Epoch [45][100/201] lr: 1.000e-02, eta: 0:15:11, time: 0.089, data_time: 0.035, memory: 987, student.loss: 0.1486, distiller.distance_wise_loss.0: 0.0701, distiller.angle_wise_loss.0: 0.1758, loss: 0.3945 2022-05-06 14:25:44,485 - mmcls - INFO - Epoch [45][200/201] lr: 1.000e-02, eta: 0:15:03, time: 0.086, data_time: 0.027, memory: 987, student.loss: 0.1522, distiller.distance_wise_loss.0: 0.0754, distiller.angle_wise_loss.0: 0.1834, loss: 0.4109 2022-05-06 14:25:44,572 - mmcls - INFO - Saving checkpoint at 45 epochs 2022-05-06 14:25:50,046 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:25:50,046 - mmcls - INFO - Epoch(val) [45][51] accuracy_top-1: 96.0025 2022-05-06 14:25:59,964 - mmcls - INFO - Epoch [46][100/201] lr: 1.000e-02, eta: 0:14:56, time: 0.088, data_time: 0.041, memory: 987, student.loss: 0.1518, distiller.distance_wise_loss.0: 0.0675, distiller.angle_wise_loss.0: 0.1701, loss: 0.3895 2022-05-06 14:26:06,571 - mmcls - INFO - Epoch [46][200/201] lr: 1.000e-02, eta: 0:14:46, time: 0.066, data_time: 0.018, memory: 987, student.loss: 0.1373, distiller.distance_wise_loss.0: 0.0708, distiller.angle_wise_loss.0: 0.1792, loss: 0.3873 2022-05-06 14:26:06,681 - mmcls - INFO - Saving checkpoint at 46 epochs 2022-05-06 14:26:12,897 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:26:12,897 - mmcls - INFO - Epoch(val) [46][51] accuracy_top-1: 96.1274 2022-05-06 14:26:22,648 - mmcls - INFO - Epoch [47][100/201] lr: 1.000e-02, eta: 0:14:39, time: 0.093, data_time: 0.044, memory: 987, student.loss: 0.1480, distiller.distance_wise_loss.0: 0.0676, distiller.angle_wise_loss.0: 0.1698, loss: 0.3854 2022-05-06 14:26:30,425 - mmcls - INFO - Epoch [47][200/201] lr: 1.000e-02, eta: 0:14:30, time: 0.078, data_time: 0.028, memory: 987, student.loss: 0.1593, distiller.distance_wise_loss.0: 0.0686, distiller.angle_wise_loss.0: 0.1730, loss: 0.4009 2022-05-06 14:26:30,530 - mmcls - INFO - Saving checkpoint at 47 epochs 2022-05-06 14:26:36,860 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:26:36,860 - mmcls - INFO - Epoch(val) [47][51] accuracy_top-1: 96.1899 2022-05-06 14:26:45,558 - mmcls - INFO - Epoch [48][100/201] lr: 1.000e-02, eta: 0:14:22, time: 0.086, data_time: 0.037, memory: 987, student.loss: 0.1447, distiller.distance_wise_loss.0: 0.0656, distiller.angle_wise_loss.0: 0.1677, loss: 0.3779 2022-05-06 14:26:52,305 - mmcls - INFO - Epoch [48][200/201] lr: 1.000e-02, eta: 0:14:12, time: 0.067, data_time: 0.017, memory: 987, student.loss: 0.1553, distiller.distance_wise_loss.0: 0.0733, distiller.angle_wise_loss.0: 0.1784, loss: 0.4070 2022-05-06 14:26:52,835 - mmcls - INFO - Saving checkpoint at 48 epochs 2022-05-06 14:26:58,337 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:26:58,338 - mmcls - INFO - Epoch(val) [48][51] accuracy_top-1: 96.3773 2022-05-06 14:27:07,198 - mmcls - INFO - Epoch [49][100/201] lr: 1.000e-02, eta: 0:14:05, time: 0.088, data_time: 0.040, memory: 987, student.loss: 0.1509, distiller.distance_wise_loss.0: 0.0687, distiller.angle_wise_loss.0: 0.1704, loss: 0.3900 2022-05-06 14:27:14,188 - mmcls - INFO - Epoch [49][200/201] lr: 1.000e-02, eta: 0:13:55, time: 0.070, data_time: 0.022, memory: 987, student.loss: 0.1479, distiller.distance_wise_loss.0: 0.0711, distiller.angle_wise_loss.0: 0.1750, loss: 0.3940 2022-05-06 14:27:14,296 - mmcls - INFO - Saving checkpoint at 49 epochs 2022-05-06 14:27:20,420 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:27:20,420 - mmcls - INFO - Epoch(val) [49][51] accuracy_top-1: 95.7527 2022-05-06 14:27:29,789 - mmcls - INFO - Epoch [50][100/201] lr: 1.000e-02, eta: 0:13:48, time: 0.093, data_time: 0.043, memory: 987, student.loss: 0.1520, distiller.distance_wise_loss.0: 0.0680, distiller.angle_wise_loss.0: 0.1704, loss: 0.3903 2022-05-06 14:27:37,628 - mmcls - INFO - Epoch [50][200/201] lr: 1.000e-02, eta: 0:13:40, time: 0.078, data_time: 0.013, memory: 987, student.loss: 0.1575, distiller.distance_wise_loss.0: 0.0677, distiller.angle_wise_loss.0: 0.1681, loss: 0.3933 2022-05-06 14:27:37,719 - mmcls - INFO - Saving checkpoint at 50 epochs 2022-05-06 14:27:43,745 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:27:43,746 - mmcls - INFO - Epoch(val) [50][51] accuracy_top-1: 96.0650 2022-05-06 14:27:53,498 - mmcls - INFO - Epoch [51][100/201] lr: 1.000e-02, eta: 0:13:33, time: 0.097, data_time: 0.043, memory: 987, student.loss: 0.1391, distiller.distance_wise_loss.0: 0.0666, distiller.angle_wise_loss.0: 0.1650, loss: 0.3708 2022-05-06 14:28:00,338 - mmcls - INFO - Epoch [51][200/201] lr: 1.000e-02, eta: 0:13:23, time: 0.069, data_time: 0.012, memory: 987, student.loss: 0.1497, distiller.distance_wise_loss.0: 0.0693, distiller.angle_wise_loss.0: 0.1712, loss: 0.3902 2022-05-06 14:28:00,812 - mmcls - INFO - Saving checkpoint at 51 epochs 2022-05-06 14:28:07,677 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:28:07,677 - mmcls - INFO - Epoch(val) [51][51] accuracy_top-1: 95.9400 2022-05-06 14:28:17,637 - mmcls - INFO - Epoch [52][100/201] lr: 1.000e-02, eta: 0:13:16, time: 0.088, data_time: 0.040, memory: 987, student.loss: 0.1431, distiller.distance_wise_loss.0: 0.0660, distiller.angle_wise_loss.0: 0.1651, loss: 0.3743 2022-05-06 14:28:24,824 - mmcls - INFO - Epoch [52][200/201] lr: 1.000e-02, eta: 0:13:07, time: 0.072, data_time: 0.021, memory: 987, student.loss: 0.1425, distiller.distance_wise_loss.0: 0.0685, distiller.angle_wise_loss.0: 0.1717, loss: 0.3828 2022-05-06 14:28:24,911 - mmcls - INFO - Saving checkpoint at 52 epochs 2022-05-06 14:28:30,947 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:28:30,948 - mmcls - INFO - Epoch(val) [52][51] accuracy_top-1: 95.8776 2022-05-06 14:28:40,744 - mmcls - INFO - Epoch [53][100/201] lr: 1.000e-02, eta: 0:12:59, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.1493, distiller.distance_wise_loss.0: 0.0664, distiller.angle_wise_loss.0: 0.1666, loss: 0.3824 2022-05-06 14:28:47,749 - mmcls - INFO - Epoch [53][200/201] lr: 1.000e-02, eta: 0:12:50, time: 0.070, data_time: 0.021, memory: 987, student.loss: 0.1413, distiller.distance_wise_loss.0: 0.0672, distiller.angle_wise_loss.0: 0.1690, loss: 0.3775 2022-05-06 14:28:47,853 - mmcls - INFO - Saving checkpoint at 53 epochs 2022-05-06 14:28:54,461 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:28:54,461 - mmcls - INFO - Epoch(val) [53][51] accuracy_top-1: 96.0650 2022-05-06 14:29:03,570 - mmcls - INFO - Epoch [54][100/201] lr: 1.000e-02, eta: 0:12:42, time: 0.091, data_time: 0.042, memory: 987, student.loss: 0.1427, distiller.distance_wise_loss.0: 0.0667, distiller.angle_wise_loss.0: 0.1672, loss: 0.3766 2022-05-06 14:29:11,051 - mmcls - INFO - Epoch [54][200/201] lr: 1.000e-02, eta: 0:12:34, time: 0.075, data_time: 0.026, memory: 987, student.loss: 0.1529, distiller.distance_wise_loss.0: 0.0705, distiller.angle_wise_loss.0: 0.1768, loss: 0.4002 2022-05-06 14:29:11,797 - mmcls - INFO - Saving checkpoint at 54 epochs 2022-05-06 14:29:18,174 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:29:18,174 - mmcls - INFO - Epoch(val) [54][51] accuracy_top-1: 95.9400 2022-05-06 14:29:27,569 - mmcls - INFO - Epoch [55][100/201] lr: 1.000e-02, eta: 0:12:26, time: 0.093, data_time: 0.047, memory: 987, student.loss: 0.1486, distiller.distance_wise_loss.0: 0.0650, distiller.angle_wise_loss.0: 0.1646, loss: 0.3782 2022-05-06 14:29:34,704 - mmcls - INFO - Epoch [55][200/201] lr: 1.000e-02, eta: 0:12:17, time: 0.071, data_time: 0.020, memory: 987, student.loss: 0.1503, distiller.distance_wise_loss.0: 0.0642, distiller.angle_wise_loss.0: 0.1616, loss: 0.3760 2022-05-06 14:29:35,797 - mmcls - INFO - Saving checkpoint at 55 epochs 2022-05-06 14:29:42,519 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:29:42,520 - mmcls - INFO - Epoch(val) [55][51] accuracy_top-1: 96.3773 2022-05-06 14:29:50,999 - mmcls - INFO - Epoch [56][100/201] lr: 1.000e-02, eta: 0:12:09, time: 0.084, data_time: 0.036, memory: 987, student.loss: 0.1582, distiller.distance_wise_loss.0: 0.0664, distiller.angle_wise_loss.0: 0.1645, loss: 0.3891 2022-05-06 14:29:57,836 - mmcls - INFO - Epoch [56][200/201] lr: 1.000e-02, eta: 0:12:00, time: 0.068, data_time: 0.021, memory: 987, student.loss: 0.1493, distiller.distance_wise_loss.0: 0.0657, distiller.angle_wise_loss.0: 0.1639, loss: 0.3789 2022-05-06 14:29:57,947 - mmcls - INFO - Saving checkpoint at 56 epochs 2022-05-06 14:30:04,743 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:30:04,743 - mmcls - INFO - Epoch(val) [56][51] accuracy_top-1: 96.1899 2022-05-06 14:30:13,885 - mmcls - INFO - Epoch [57][100/201] lr: 1.000e-02, eta: 0:11:53, time: 0.091, data_time: 0.042, memory: 987, student.loss: 0.1354, distiller.distance_wise_loss.0: 0.0640, distiller.angle_wise_loss.0: 0.1628, loss: 0.3623 2022-05-06 14:30:21,627 - mmcls - INFO - Epoch [57][200/201] lr: 1.000e-02, eta: 0:11:44, time: 0.077, data_time: 0.029, memory: 987, student.loss: 0.1391, distiller.distance_wise_loss.0: 0.0625, distiller.angle_wise_loss.0: 0.1585, loss: 0.3601 2022-05-06 14:30:21,720 - mmcls - INFO - Saving checkpoint at 57 epochs 2022-05-06 14:30:27,187 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:30:27,187 - mmcls - INFO - Epoch(val) [57][51] accuracy_top-1: 96.1274 2022-05-06 14:30:36,264 - mmcls - INFO - Epoch [58][100/201] lr: 1.000e-02, eta: 0:11:36, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.1453, distiller.distance_wise_loss.0: 0.0648, distiller.angle_wise_loss.0: 0.1639, loss: 0.3741 2022-05-06 14:30:43,256 - mmcls - INFO - Epoch [58][200/201] lr: 1.000e-02, eta: 0:11:27, time: 0.070, data_time: 0.021, memory: 987, student.loss: 0.1389, distiller.distance_wise_loss.0: 0.0598, distiller.angle_wise_loss.0: 0.1524, loss: 0.3510 2022-05-06 14:30:43,343 - mmcls - INFO - Saving checkpoint at 58 epochs 2022-05-06 14:30:49,587 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:30:49,587 - mmcls - INFO - Epoch(val) [58][51] accuracy_top-1: 96.4397 2022-05-06 14:30:58,608 - mmcls - INFO - Epoch [59][100/201] lr: 1.000e-02, eta: 0:11:20, time: 0.090, data_time: 0.042, memory: 987, student.loss: 0.1436, distiller.distance_wise_loss.0: 0.0594, distiller.angle_wise_loss.0: 0.1514, loss: 0.3544 2022-05-06 14:31:05,131 - mmcls - INFO - Epoch [59][200/201] lr: 1.000e-02, eta: 0:11:10, time: 0.065, data_time: 0.016, memory: 987, student.loss: 0.1450, distiller.distance_wise_loss.0: 0.0659, distiller.angle_wise_loss.0: 0.1627, loss: 0.3736 2022-05-06 14:31:05,888 - mmcls - INFO - Saving checkpoint at 59 epochs 2022-05-06 14:31:12,519 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:31:12,519 - mmcls - INFO - Epoch(val) [59][51] accuracy_top-1: 96.0650 2022-05-06 14:31:21,334 - mmcls - INFO - Epoch [60][100/201] lr: 1.000e-02, eta: 0:11:03, time: 0.088, data_time: 0.039, memory: 987, student.loss: 0.1411, distiller.distance_wise_loss.0: 0.0593, distiller.angle_wise_loss.0: 0.1515, loss: 0.3519 2022-05-06 14:31:28,775 - mmcls - INFO - Epoch [60][200/201] lr: 1.000e-02, eta: 0:10:54, time: 0.074, data_time: 0.017, memory: 987, student.loss: 0.1475, distiller.distance_wise_loss.0: 0.0627, distiller.angle_wise_loss.0: 0.1581, loss: 0.3684 2022-05-06 14:31:29,255 - mmcls - INFO - Saving checkpoint at 60 epochs 2022-05-06 14:31:35,336 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:31:35,337 - mmcls - INFO - Epoch(val) [60][51] accuracy_top-1: 96.2523 2022-05-06 14:31:44,358 - mmcls - INFO - Epoch [61][100/201] lr: 1.000e-03, eta: 0:10:46, time: 0.089, data_time: 0.042, memory: 987, student.loss: 0.1426, distiller.distance_wise_loss.0: 0.0642, distiller.angle_wise_loss.0: 0.1600, loss: 0.3667 2022-05-06 14:31:51,832 - mmcls - INFO - Epoch [61][200/201] lr: 1.000e-03, eta: 0:10:38, time: 0.075, data_time: 0.027, memory: 987, student.loss: 0.1314, distiller.distance_wise_loss.0: 0.0624, distiller.angle_wise_loss.0: 0.1565, loss: 0.3504 2022-05-06 14:31:52,125 - mmcls - INFO - Saving checkpoint at 61 epochs 2022-05-06 14:31:57,297 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:31:57,298 - mmcls - INFO - Epoch(val) [61][51] accuracy_top-1: 96.3148 2022-05-06 14:32:07,053 - mmcls - INFO - Epoch [62][100/201] lr: 1.000e-03, eta: 0:10:30, time: 0.089, data_time: 0.041, memory: 987, student.loss: 0.1311, distiller.distance_wise_loss.0: 0.0598, distiller.angle_wise_loss.0: 0.1532, loss: 0.3441 2022-05-06 14:32:14,014 - mmcls - INFO - Epoch [62][200/201] lr: 1.000e-03, eta: 0:10:21, time: 0.070, data_time: 0.022, memory: 987, student.loss: 0.1532, distiller.distance_wise_loss.0: 0.0579, distiller.angle_wise_loss.0: 0.1472, loss: 0.3584 2022-05-06 14:32:14,514 - mmcls - INFO - Saving checkpoint at 62 epochs 2022-05-06 14:32:20,784 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:32:20,785 - mmcls - INFO - Epoch(val) [62][51] accuracy_top-1: 96.1274 2022-05-06 14:32:30,763 - mmcls - INFO - Epoch [63][100/201] lr: 1.000e-03, eta: 0:10:14, time: 0.099, data_time: 0.037, memory: 987, student.loss: 0.1389, distiller.distance_wise_loss.0: 0.0598, distiller.angle_wise_loss.0: 0.1535, loss: 0.3522 2022-05-06 14:32:38,709 - mmcls - INFO - Epoch [63][200/201] lr: 1.000e-03, eta: 0:10:05, time: 0.079, data_time: 0.015, memory: 987, student.loss: 0.1442, distiller.distance_wise_loss.0: 0.0593, distiller.angle_wise_loss.0: 0.1534, loss: 0.3569 2022-05-06 14:32:39,396 - mmcls - INFO - Saving checkpoint at 63 epochs 2022-05-06 14:32:45,894 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:32:45,895 - mmcls - INFO - Epoch(val) [63][51] accuracy_top-1: 96.0025 2022-05-06 14:32:55,754 - mmcls - INFO - Epoch [64][100/201] lr: 1.000e-03, eta: 0:09:58, time: 0.089, data_time: 0.039, memory: 987, student.loss: 0.1396, distiller.distance_wise_loss.0: 0.0619, distiller.angle_wise_loss.0: 0.1562, loss: 0.3578 2022-05-06 14:33:02,370 - mmcls - INFO - Epoch [64][200/201] lr: 1.000e-03, eta: 0:09:49, time: 0.066, data_time: 0.015, memory: 987, student.loss: 0.1376, distiller.distance_wise_loss.0: 0.0556, distiller.angle_wise_loss.0: 0.1438, loss: 0.3370 2022-05-06 14:33:03,016 - mmcls - INFO - Saving checkpoint at 64 epochs 2022-05-06 14:33:09,530 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:33:09,531 - mmcls - INFO - Epoch(val) [64][51] accuracy_top-1: 96.1899 2022-05-06 14:33:18,044 - mmcls - INFO - Epoch [65][100/201] lr: 1.000e-03, eta: 0:09:41, time: 0.085, data_time: 0.034, memory: 987, student.loss: 0.1437, distiller.distance_wise_loss.0: 0.0612, distiller.angle_wise_loss.0: 0.1526, loss: 0.3576 2022-05-06 14:33:25,315 - mmcls - INFO - Epoch [65][200/201] lr: 1.000e-03, eta: 0:09:32, time: 0.073, data_time: 0.017, memory: 987, student.loss: 0.1449, distiller.distance_wise_loss.0: 0.0609, distiller.angle_wise_loss.0: 0.1510, loss: 0.3567 2022-05-06 14:33:25,416 - mmcls - INFO - Saving checkpoint at 65 epochs 2022-05-06 14:33:32,112 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:33:32,112 - mmcls - INFO - Epoch(val) [65][51] accuracy_top-1: 96.2523 2022-05-06 14:33:41,548 - mmcls - INFO - Epoch [66][100/201] lr: 1.000e-03, eta: 0:09:24, time: 0.094, data_time: 0.045, memory: 987, student.loss: 0.1270, distiller.distance_wise_loss.0: 0.0566, distiller.angle_wise_loss.0: 0.1452, loss: 0.3288 2022-05-06 14:33:48,420 - mmcls - INFO - Epoch [66][200/201] lr: 1.000e-03, eta: 0:09:16, time: 0.069, data_time: 0.020, memory: 987, student.loss: 0.1418, distiller.distance_wise_loss.0: 0.0606, distiller.angle_wise_loss.0: 0.1529, loss: 0.3553 2022-05-06 14:33:48,497 - mmcls - INFO - Saving checkpoint at 66 epochs 2022-05-06 14:33:55,406 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:33:55,407 - mmcls - INFO - Epoch(val) [66][51] accuracy_top-1: 96.3148 2022-05-06 14:34:05,340 - mmcls - INFO - Epoch [67][100/201] lr: 1.000e-03, eta: 0:09:08, time: 0.092, data_time: 0.043, memory: 987, student.loss: 0.1342, distiller.distance_wise_loss.0: 0.0597, distiller.angle_wise_loss.0: 0.1500, loss: 0.3438 2022-05-06 14:34:12,558 - mmcls - INFO - Epoch [67][200/201] lr: 1.000e-03, eta: 0:08:59, time: 0.072, data_time: 0.021, memory: 987, student.loss: 0.1332, distiller.distance_wise_loss.0: 0.0578, distiller.angle_wise_loss.0: 0.1495, loss: 0.3405 2022-05-06 14:34:12,652 - mmcls - INFO - Saving checkpoint at 67 epochs 2022-05-06 14:34:18,291 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:34:18,292 - mmcls - INFO - Epoch(val) [67][51] accuracy_top-1: 96.1274 2022-05-06 14:34:27,795 - mmcls - INFO - Epoch [68][100/201] lr: 1.000e-03, eta: 0:08:52, time: 0.095, data_time: 0.034, memory: 987, student.loss: 0.1432, distiller.distance_wise_loss.0: 0.0603, distiller.angle_wise_loss.0: 0.1511, loss: 0.3546 2022-05-06 14:34:35,120 - mmcls - INFO - Epoch [68][200/201] lr: 1.000e-03, eta: 0:08:43, time: 0.073, data_time: 0.026, memory: 987, student.loss: 0.1410, distiller.distance_wise_loss.0: 0.0591, distiller.angle_wise_loss.0: 0.1501, loss: 0.3501 2022-05-06 14:34:35,368 - mmcls - INFO - Saving checkpoint at 68 epochs 2022-05-06 14:34:42,409 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:34:42,410 - mmcls - INFO - Epoch(val) [68][51] accuracy_top-1: 96.0650 2022-05-06 14:34:51,415 - mmcls - INFO - Epoch [69][100/201] lr: 1.000e-03, eta: 0:08:35, time: 0.090, data_time: 0.042, memory: 987, student.loss: 0.1490, distiller.distance_wise_loss.0: 0.0579, distiller.angle_wise_loss.0: 0.1488, loss: 0.3558 2022-05-06 14:34:58,698 - mmcls - INFO - Epoch [69][200/201] lr: 1.000e-03, eta: 0:08:27, time: 0.073, data_time: 0.024, memory: 987, student.loss: 0.1348, distiller.distance_wise_loss.0: 0.0577, distiller.angle_wise_loss.0: 0.1452, loss: 0.3377 2022-05-06 14:34:59,567 - mmcls - INFO - Saving checkpoint at 69 epochs 2022-05-06 14:35:05,566 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:35:05,567 - mmcls - INFO - Epoch(val) [69][51] accuracy_top-1: 96.1274 2022-05-06 14:35:15,408 - mmcls - INFO - Epoch [70][100/201] lr: 1.000e-03, eta: 0:08:19, time: 0.098, data_time: 0.035, memory: 987, student.loss: 0.1413, distiller.distance_wise_loss.0: 0.0628, distiller.angle_wise_loss.0: 0.1575, loss: 0.3616 2022-05-06 14:35:22,186 - mmcls - INFO - Epoch [70][200/201] lr: 1.000e-03, eta: 0:08:10, time: 0.068, data_time: 0.019, memory: 987, student.loss: 0.1479, distiller.distance_wise_loss.0: 0.0586, distiller.angle_wise_loss.0: 0.1515, loss: 0.3580 2022-05-06 14:35:22,314 - mmcls - INFO - Saving checkpoint at 70 epochs 2022-05-06 14:35:27,872 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:35:27,873 - mmcls - INFO - Epoch(val) [70][51] accuracy_top-1: 96.3148 2022-05-06 14:35:37,096 - mmcls - INFO - Epoch [71][100/201] lr: 1.000e-03, eta: 0:08:03, time: 0.092, data_time: 0.042, memory: 987, student.loss: 0.1501, distiller.distance_wise_loss.0: 0.0578, distiller.angle_wise_loss.0: 0.1496, loss: 0.3574 2022-05-06 14:35:45,550 - mmcls - INFO - Epoch [71][200/201] lr: 1.000e-03, eta: 0:07:55, time: 0.085, data_time: 0.035, memory: 987, student.loss: 0.1361, distiller.distance_wise_loss.0: 0.0596, distiller.angle_wise_loss.0: 0.1495, loss: 0.3451 2022-05-06 14:35:45,640 - mmcls - INFO - Saving checkpoint at 71 epochs 2022-05-06 14:35:52,617 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:35:52,618 - mmcls - INFO - Epoch(val) [71][51] accuracy_top-1: 96.5022 2022-05-06 14:36:01,736 - mmcls - INFO - Epoch [72][100/201] lr: 1.000e-03, eta: 0:07:47, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.1463, distiller.distance_wise_loss.0: 0.0610, distiller.angle_wise_loss.0: 0.1538, loss: 0.3610 2022-05-06 14:36:08,821 - mmcls - INFO - Epoch [72][200/201] lr: 1.000e-03, eta: 0:07:38, time: 0.071, data_time: 0.016, memory: 987, student.loss: 0.1401, distiller.distance_wise_loss.0: 0.0585, distiller.angle_wise_loss.0: 0.1460, loss: 0.3446 2022-05-06 14:36:09,496 - mmcls - INFO - Saving checkpoint at 72 epochs 2022-05-06 14:36:16,221 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:36:16,222 - mmcls - INFO - Epoch(val) [72][51] accuracy_top-1: 95.8151 2022-05-06 14:36:26,504 - mmcls - INFO - Epoch [73][100/201] lr: 1.000e-03, eta: 0:07:30, time: 0.089, data_time: 0.041, memory: 987, student.loss: 0.1358, distiller.distance_wise_loss.0: 0.0587, distiller.angle_wise_loss.0: 0.1482, loss: 0.3427 2022-05-06 14:36:33,572 - mmcls - INFO - Epoch [73][200/201] lr: 1.000e-03, eta: 0:07:22, time: 0.071, data_time: 0.015, memory: 987, student.loss: 0.1303, distiller.distance_wise_loss.0: 0.0582, distiller.angle_wise_loss.0: 0.1491, loss: 0.3376 2022-05-06 14:36:34,292 - mmcls - INFO - Saving checkpoint at 73 epochs 2022-05-06 14:36:41,107 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:36:41,107 - mmcls - INFO - Epoch(val) [73][51] accuracy_top-1: 96.2523 2022-05-06 14:36:51,348 - mmcls - INFO - Epoch [74][100/201] lr: 1.000e-03, eta: 0:07:14, time: 0.092, data_time: 0.042, memory: 987, student.loss: 0.1381, distiller.distance_wise_loss.0: 0.0573, distiller.angle_wise_loss.0: 0.1456, loss: 0.3410 2022-05-06 14:36:59,088 - mmcls - INFO - Epoch [74][200/201] lr: 1.000e-03, eta: 0:07:05, time: 0.077, data_time: 0.015, memory: 987, student.loss: 0.1360, distiller.distance_wise_loss.0: 0.0594, distiller.angle_wise_loss.0: 0.1506, loss: 0.3460 2022-05-06 14:36:59,890 - mmcls - INFO - Saving checkpoint at 74 epochs 2022-05-06 14:37:05,339 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:37:05,339 - mmcls - INFO - Epoch(val) [74][51] accuracy_top-1: 96.1899 2022-05-06 14:37:15,092 - mmcls - INFO - Epoch [75][100/201] lr: 1.000e-03, eta: 0:06:57, time: 0.091, data_time: 0.043, memory: 987, student.loss: 0.1378, distiller.distance_wise_loss.0: 0.0567, distiller.angle_wise_loss.0: 0.1432, loss: 0.3377 2022-05-06 14:37:21,946 - mmcls - INFO - Epoch [75][200/201] lr: 1.000e-03, eta: 0:06:49, time: 0.069, data_time: 0.020, memory: 987, student.loss: 0.1454, distiller.distance_wise_loss.0: 0.0596, distiller.angle_wise_loss.0: 0.1517, loss: 0.3566 2022-05-06 14:37:22,186 - mmcls - INFO - Saving checkpoint at 75 epochs 2022-05-06 14:37:28,950 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:37:28,951 - mmcls - INFO - Epoch(val) [75][51] accuracy_top-1: 96.2523 2022-05-06 14:37:39,150 - mmcls - INFO - Epoch [76][100/201] lr: 1.000e-03, eta: 0:06:41, time: 0.090, data_time: 0.043, memory: 987, student.loss: 0.1410, distiller.distance_wise_loss.0: 0.0592, distiller.angle_wise_loss.0: 0.1487, loss: 0.3489 2022-05-06 14:37:46,274 - mmcls - INFO - Epoch [76][200/201] lr: 1.000e-03, eta: 0:06:32, time: 0.071, data_time: 0.022, memory: 987, student.loss: 0.1481, distiller.distance_wise_loss.0: 0.0557, distiller.angle_wise_loss.0: 0.1448, loss: 0.3486 2022-05-06 14:37:46,366 - mmcls - INFO - Saving checkpoint at 76 epochs 2022-05-06 14:37:52,157 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:37:52,157 - mmcls - INFO - Epoch(val) [76][51] accuracy_top-1: 96.3773 2022-05-06 14:38:01,303 - mmcls - INFO - Epoch [77][100/201] lr: 1.000e-03, eta: 0:06:24, time: 0.091, data_time: 0.042, memory: 987, student.loss: 0.1451, distiller.distance_wise_loss.0: 0.0581, distiller.angle_wise_loss.0: 0.1471, loss: 0.3503 2022-05-06 14:38:09,593 - mmcls - INFO - Epoch [77][200/201] lr: 1.000e-03, eta: 0:06:16, time: 0.083, data_time: 0.016, memory: 987, student.loss: 0.1408, distiller.distance_wise_loss.0: 0.0584, distiller.angle_wise_loss.0: 0.1482, loss: 0.3474 2022-05-06 14:38:11,201 - mmcls - INFO - Saving checkpoint at 77 epochs 2022-05-06 14:38:18,089 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:38:18,090 - mmcls - INFO - Epoch(val) [77][51] accuracy_top-1: 96.3148 2022-05-06 14:38:28,179 - mmcls - INFO - Epoch [78][100/201] lr: 1.000e-03, eta: 0:06:08, time: 0.087, data_time: 0.039, memory: 987, student.loss: 0.1381, distiller.distance_wise_loss.0: 0.0607, distiller.angle_wise_loss.0: 0.1562, loss: 0.3550 2022-05-06 14:38:34,831 - mmcls - INFO - Epoch [78][200/201] lr: 1.000e-03, eta: 0:06:00, time: 0.067, data_time: 0.019, memory: 987, student.loss: 0.1429, distiller.distance_wise_loss.0: 0.0565, distiller.angle_wise_loss.0: 0.1455, loss: 0.3449 2022-05-06 14:38:34,919 - mmcls - INFO - Saving checkpoint at 78 epochs 2022-05-06 14:38:41,687 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:38:41,688 - mmcls - INFO - Epoch(val) [78][51] accuracy_top-1: 96.1274 2022-05-06 14:38:51,007 - mmcls - INFO - Epoch [79][100/201] lr: 1.000e-03, eta: 0:05:52, time: 0.093, data_time: 0.042, memory: 987, student.loss: 0.1399, distiller.distance_wise_loss.0: 0.0561, distiller.angle_wise_loss.0: 0.1437, loss: 0.3398 2022-05-06 14:38:57,924 - mmcls - INFO - Epoch [79][200/201] lr: 1.000e-03, eta: 0:05:43, time: 0.069, data_time: 0.014, memory: 987, student.loss: 0.1274, distiller.distance_wise_loss.0: 0.0600, distiller.angle_wise_loss.0: 0.1505, loss: 0.3379 2022-05-06 14:38:58,080 - mmcls - INFO - Saving checkpoint at 79 epochs 2022-05-06 14:39:04,109 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:39:04,109 - mmcls - INFO - Epoch(val) [79][51] accuracy_top-1: 96.2523 2022-05-06 14:39:13,573 - mmcls - INFO - Epoch [80][100/201] lr: 1.000e-03, eta: 0:05:35, time: 0.094, data_time: 0.043, memory: 987, student.loss: 0.1351, distiller.distance_wise_loss.0: 0.0581, distiller.angle_wise_loss.0: 0.1476, loss: 0.3409 2022-05-06 14:39:22,689 - mmcls - INFO - Epoch [80][200/201] lr: 1.000e-03, eta: 0:05:27, time: 0.091, data_time: 0.041, memory: 987, student.loss: 0.1347, distiller.distance_wise_loss.0: 0.0584, distiller.angle_wise_loss.0: 0.1491, loss: 0.3422 2022-05-06 14:39:24,283 - mmcls - INFO - Saving checkpoint at 80 epochs 2022-05-06 14:39:31,172 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:39:31,173 - mmcls - INFO - Epoch(val) [80][51] accuracy_top-1: 96.0650 2022-05-06 14:39:41,211 - mmcls - INFO - Epoch [81][100/201] lr: 1.000e-03, eta: 0:05:20, time: 0.095, data_time: 0.044, memory: 987, student.loss: 0.1377, distiller.distance_wise_loss.0: 0.0597, distiller.angle_wise_loss.0: 0.1508, loss: 0.3481 2022-05-06 14:39:48,811 - mmcls - INFO - Epoch [81][200/201] lr: 1.000e-03, eta: 0:05:11, time: 0.076, data_time: 0.028, memory: 987, student.loss: 0.1363, distiller.distance_wise_loss.0: 0.0545, distiller.angle_wise_loss.0: 0.1420, loss: 0.3328 2022-05-06 14:39:48,922 - mmcls - INFO - Saving checkpoint at 81 epochs 2022-05-06 14:39:55,710 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:39:55,711 - mmcls - INFO - Epoch(val) [81][51] accuracy_top-1: 96.1899 2022-05-06 14:40:04,887 - mmcls - INFO - Epoch [82][100/201] lr: 1.000e-03, eta: 0:05:03, time: 0.089, data_time: 0.039, memory: 987, student.loss: 0.1341, distiller.distance_wise_loss.0: 0.0595, distiller.angle_wise_loss.0: 0.1510, loss: 0.3445 2022-05-06 14:40:11,620 - mmcls - INFO - Epoch [82][200/201] lr: 1.000e-03, eta: 0:04:55, time: 0.067, data_time: 0.019, memory: 987, student.loss: 0.1453, distiller.distance_wise_loss.0: 0.0587, distiller.angle_wise_loss.0: 0.1494, loss: 0.3534 2022-05-06 14:40:11,703 - mmcls - INFO - Saving checkpoint at 82 epochs 2022-05-06 14:40:16,840 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:40:16,840 - mmcls - INFO - Epoch(val) [82][51] accuracy_top-1: 96.3773 2022-05-06 14:40:26,759 - mmcls - INFO - Epoch [83][100/201] lr: 1.000e-03, eta: 0:04:47, time: 0.090, data_time: 0.041, memory: 987, student.loss: 0.1429, distiller.distance_wise_loss.0: 0.0578, distiller.angle_wise_loss.0: 0.1486, loss: 0.3493 2022-05-06 14:40:33,596 - mmcls - INFO - Epoch [83][200/201] lr: 1.000e-03, eta: 0:04:38, time: 0.068, data_time: 0.017, memory: 987, student.loss: 0.1388, distiller.distance_wise_loss.0: 0.0589, distiller.angle_wise_loss.0: 0.1492, loss: 0.3469 2022-05-06 14:40:34,719 - mmcls - INFO - Saving checkpoint at 83 epochs 2022-05-06 14:40:43,181 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:40:43,182 - mmcls - INFO - Epoch(val) [83][51] accuracy_top-1: 96.1899 2022-05-06 14:40:52,346 - mmcls - INFO - Epoch [84][100/201] lr: 1.000e-03, eta: 0:04:30, time: 0.091, data_time: 0.041, memory: 987, student.loss: 0.1519, distiller.distance_wise_loss.0: 0.0568, distiller.angle_wise_loss.0: 0.1450, loss: 0.3537 2022-05-06 14:40:59,900 - mmcls - INFO - Epoch [84][200/201] lr: 1.000e-03, eta: 0:04:22, time: 0.076, data_time: 0.015, memory: 987, student.loss: 0.1435, distiller.distance_wise_loss.0: 0.0572, distiller.angle_wise_loss.0: 0.1455, loss: 0.3463 2022-05-06 14:41:00,021 - mmcls - INFO - Saving checkpoint at 84 epochs 2022-05-06 14:41:06,120 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:41:06,121 - mmcls - INFO - Epoch(val) [84][51] accuracy_top-1: 96.1899 2022-05-06 14:41:16,584 - mmcls - INFO - Epoch [85][100/201] lr: 1.000e-03, eta: 0:04:14, time: 0.100, data_time: 0.040, memory: 987, student.loss: 0.1569, distiller.distance_wise_loss.0: 0.0569, distiller.angle_wise_loss.0: 0.1436, loss: 0.3573 2022-05-06 14:41:23,921 - mmcls - INFO - Epoch [85][200/201] lr: 1.000e-03, eta: 0:04:06, time: 0.073, data_time: 0.022, memory: 987, student.loss: 0.1411, distiller.distance_wise_loss.0: 0.0574, distiller.angle_wise_loss.0: 0.1453, loss: 0.3439 2022-05-06 14:41:24,979 - mmcls - INFO - Saving checkpoint at 85 epochs 2022-05-06 14:41:31,677 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:41:31,678 - mmcls - INFO - Epoch(val) [85][51] accuracy_top-1: 96.0650 2022-05-06 14:41:40,732 - mmcls - INFO - Epoch [86][100/201] lr: 1.000e-03, eta: 0:03:58, time: 0.090, data_time: 0.040, memory: 987, student.loss: 0.1329, distiller.distance_wise_loss.0: 0.0599, distiller.angle_wise_loss.0: 0.1512, loss: 0.3440 2022-05-06 14:41:49,121 - mmcls - INFO - Epoch [86][200/201] lr: 1.000e-03, eta: 0:03:49, time: 0.084, data_time: 0.034, memory: 987, student.loss: 0.1341, distiller.distance_wise_loss.0: 0.0585, distiller.angle_wise_loss.0: 0.1464, loss: 0.3390 2022-05-06 14:41:49,875 - mmcls - INFO - Saving checkpoint at 86 epochs 2022-05-06 14:41:55,507 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:41:55,508 - mmcls - INFO - Epoch(val) [86][51] accuracy_top-1: 96.1899 2022-05-06 14:42:04,440 - mmcls - INFO - Epoch [87][100/201] lr: 1.000e-03, eta: 0:03:41, time: 0.089, data_time: 0.040, memory: 987, student.loss: 0.1453, distiller.distance_wise_loss.0: 0.0619, distiller.angle_wise_loss.0: 0.1533, loss: 0.3604 2022-05-06 14:42:11,679 - mmcls - INFO - Epoch [87][200/201] lr: 1.000e-03, eta: 0:03:33, time: 0.072, data_time: 0.022, memory: 987, student.loss: 0.1403, distiller.distance_wise_loss.0: 0.0584, distiller.angle_wise_loss.0: 0.1485, loss: 0.3472 2022-05-06 14:42:12,963 - mmcls - INFO - Saving checkpoint at 87 epochs 2022-05-06 14:42:20,263 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:42:20,264 - mmcls - INFO - Epoch(val) [87][51] accuracy_top-1: 96.1899 2022-05-06 14:42:30,088 - mmcls - INFO - Epoch [88][100/201] lr: 1.000e-03, eta: 0:03:25, time: 0.089, data_time: 0.038, memory: 987, student.loss: 0.1372, distiller.distance_wise_loss.0: 0.0550, distiller.angle_wise_loss.0: 0.1415, loss: 0.3337 2022-05-06 14:42:37,651 - mmcls - INFO - Epoch [88][200/201] lr: 1.000e-03, eta: 0:03:17, time: 0.076, data_time: 0.026, memory: 987, student.loss: 0.1311, distiller.distance_wise_loss.0: 0.0604, distiller.angle_wise_loss.0: 0.1502, loss: 0.3418 2022-05-06 14:42:37,740 - mmcls - INFO - Saving checkpoint at 88 epochs 2022-05-06 14:42:43,303 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:42:43,303 - mmcls - INFO - Epoch(val) [88][51] accuracy_top-1: 96.2523 2022-05-06 14:42:52,982 - mmcls - INFO - Epoch [89][100/201] lr: 1.000e-03, eta: 0:03:08, time: 0.089, data_time: 0.040, memory: 987, student.loss: 0.1453, distiller.distance_wise_loss.0: 0.0591, distiller.angle_wise_loss.0: 0.1494, loss: 0.3538 2022-05-06 14:42:59,968 - mmcls - INFO - Epoch [89][200/201] lr: 1.000e-03, eta: 0:03:00, time: 0.070, data_time: 0.021, memory: 987, student.loss: 0.1306, distiller.distance_wise_loss.0: 0.0557, distiller.angle_wise_loss.0: 0.1437, loss: 0.3300 2022-05-06 14:43:00,157 - mmcls - INFO - Saving checkpoint at 89 epochs 2022-05-06 14:43:05,318 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:43:05,319 - mmcls - INFO - Epoch(val) [89][51] accuracy_top-1: 96.1899 2022-05-06 14:43:14,447 - mmcls - INFO - Epoch [90][100/201] lr: 1.000e-03, eta: 0:02:52, time: 0.091, data_time: 0.042, memory: 987, student.loss: 0.1302, distiller.distance_wise_loss.0: 0.0558, distiller.angle_wise_loss.0: 0.1440, loss: 0.3300 2022-05-06 14:43:21,618 - mmcls - INFO - Epoch [90][200/201] lr: 1.000e-03, eta: 0:02:44, time: 0.072, data_time: 0.021, memory: 987, student.loss: 0.1500, distiller.distance_wise_loss.0: 0.0574, distiller.angle_wise_loss.0: 0.1466, loss: 0.3540 2022-05-06 14:43:21,707 - mmcls - INFO - Saving checkpoint at 90 epochs 2022-05-06 14:43:27,943 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:43:27,944 - mmcls - INFO - Epoch(val) [90][51] accuracy_top-1: 96.3773 2022-05-06 14:43:37,828 - mmcls - INFO - Epoch [91][100/201] lr: 1.000e-04, eta: 0:02:35, time: 0.091, data_time: 0.043, memory: 987, student.loss: 0.1346, distiller.distance_wise_loss.0: 0.0595, distiller.angle_wise_loss.0: 0.1491, loss: 0.3432 2022-05-06 14:43:44,929 - mmcls - INFO - Epoch [91][200/201] lr: 1.000e-04, eta: 0:02:27, time: 0.071, data_time: 0.020, memory: 987, student.loss: 0.1368, distiller.distance_wise_loss.0: 0.0567, distiller.angle_wise_loss.0: 0.1447, loss: 0.3382 2022-05-06 14:43:45,047 - mmcls - INFO - Saving checkpoint at 91 epochs 2022-05-06 14:43:50,568 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:43:50,568 - mmcls - INFO - Epoch(val) [91][51] accuracy_top-1: 96.3773 2022-05-06 14:43:59,566 - mmcls - INFO - Epoch [92][100/201] lr: 1.000e-04, eta: 0:02:19, time: 0.090, data_time: 0.036, memory: 987, student.loss: 0.1482, distiller.distance_wise_loss.0: 0.0564, distiller.angle_wise_loss.0: 0.1458, loss: 0.3504 2022-05-06 14:44:07,514 - mmcls - INFO - Epoch [92][200/201] lr: 1.000e-04, eta: 0:02:11, time: 0.079, data_time: 0.021, memory: 987, student.loss: 0.1384, distiller.distance_wise_loss.0: 0.0563, distiller.angle_wise_loss.0: 0.1435, loss: 0.3382 2022-05-06 14:44:07,601 - mmcls - INFO - Saving checkpoint at 92 epochs 2022-05-06 14:44:13,197 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:44:13,197 - mmcls - INFO - Epoch(val) [92][51] accuracy_top-1: 96.1899 2022-05-06 14:44:22,175 - mmcls - INFO - Epoch [93][100/201] lr: 1.000e-04, eta: 0:02:03, time: 0.089, data_time: 0.035, memory: 987, student.loss: 0.1388, distiller.distance_wise_loss.0: 0.0583, distiller.angle_wise_loss.0: 0.1474, loss: 0.3445 2022-05-06 14:44:29,817 - mmcls - INFO - Epoch [93][200/201] lr: 1.000e-04, eta: 0:01:54, time: 0.076, data_time: 0.027, memory: 987, student.loss: 0.1342, distiller.distance_wise_loss.0: 0.0609, distiller.angle_wise_loss.0: 0.1541, loss: 0.3492 2022-05-06 14:44:31,062 - mmcls - INFO - Saving checkpoint at 93 epochs 2022-05-06 14:44:37,723 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:44:37,723 - mmcls - INFO - Epoch(val) [93][51] accuracy_top-1: 96.2523 2022-05-06 14:44:47,530 - mmcls - INFO - Epoch [94][100/201] lr: 1.000e-04, eta: 0:01:46, time: 0.098, data_time: 0.037, memory: 987, student.loss: 0.1458, distiller.distance_wise_loss.0: 0.0550, distiller.angle_wise_loss.0: 0.1397, loss: 0.3405 2022-05-06 14:44:54,406 - mmcls - INFO - Epoch [94][200/201] lr: 1.000e-04, eta: 0:01:38, time: 0.069, data_time: 0.019, memory: 987, student.loss: 0.1440, distiller.distance_wise_loss.0: 0.0587, distiller.angle_wise_loss.0: 0.1490, loss: 0.3517 2022-05-06 14:44:55,343 - mmcls - INFO - Saving checkpoint at 94 epochs 2022-05-06 14:45:01,543 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:45:01,543 - mmcls - INFO - Epoch(val) [94][51] accuracy_top-1: 96.1899 2022-05-06 14:45:10,452 - mmcls - INFO - Epoch [95][100/201] lr: 1.000e-04, eta: 0:01:30, time: 0.089, data_time: 0.039, memory: 987, student.loss: 0.1487, distiller.distance_wise_loss.0: 0.0569, distiller.angle_wise_loss.0: 0.1441, loss: 0.3497 2022-05-06 14:45:18,279 - mmcls - INFO - Epoch [95][200/201] lr: 1.000e-04, eta: 0:01:22, time: 0.078, data_time: 0.015, memory: 987, student.loss: 0.1319, distiller.distance_wise_loss.0: 0.0582, distiller.angle_wise_loss.0: 0.1497, loss: 0.3398 2022-05-06 14:45:19,331 - mmcls - INFO - Saving checkpoint at 95 epochs 2022-05-06 14:45:26,122 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:45:26,123 - mmcls - INFO - Epoch(val) [95][51] accuracy_top-1: 96.2523 2022-05-06 14:45:35,740 - mmcls - INFO - Epoch [96][100/201] lr: 1.000e-04, eta: 0:01:13, time: 0.096, data_time: 0.034, memory: 987, student.loss: 0.1466, distiller.distance_wise_loss.0: 0.0575, distiller.angle_wise_loss.0: 0.1447, loss: 0.3488 2022-05-06 14:45:42,689 - mmcls - INFO - Epoch [96][200/201] lr: 1.000e-04, eta: 0:01:05, time: 0.069, data_time: 0.020, memory: 987, student.loss: 0.1374, distiller.distance_wise_loss.0: 0.0576, distiller.angle_wise_loss.0: 0.1470, loss: 0.3420 2022-05-06 14:45:42,945 - mmcls - INFO - Saving checkpoint at 96 epochs 2022-05-06 14:45:49,877 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:45:49,878 - mmcls - INFO - Epoch(val) [96][51] accuracy_top-1: 96.4397 2022-05-06 14:45:58,977 - mmcls - INFO - Epoch [97][100/201] lr: 1.000e-04, eta: 0:00:57, time: 0.090, data_time: 0.039, memory: 987, student.loss: 0.1358, distiller.distance_wise_loss.0: 0.0592, distiller.angle_wise_loss.0: 0.1516, loss: 0.3466 2022-05-06 14:46:05,869 - mmcls - INFO - Epoch [97][200/201] lr: 1.000e-04, eta: 0:00:49, time: 0.069, data_time: 0.018, memory: 987, student.loss: 0.1444, distiller.distance_wise_loss.0: 0.0584, distiller.angle_wise_loss.0: 0.1490, loss: 0.3518 2022-05-06 14:46:05,963 - mmcls - INFO - Saving checkpoint at 97 epochs 2022-05-06 14:46:12,703 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:46:12,703 - mmcls - INFO - Epoch(val) [97][51] accuracy_top-1: 96.1899 2022-05-06 14:46:22,649 - mmcls - INFO - Epoch [98][100/201] lr: 1.000e-04, eta: 0:00:41, time: 0.091, data_time: 0.042, memory: 987, student.loss: 0.1434, distiller.distance_wise_loss.0: 0.0566, distiller.angle_wise_loss.0: 0.1426, loss: 0.3426 2022-05-06 14:46:29,625 - mmcls - INFO - Epoch [98][200/201] lr: 1.000e-04, eta: 0:00:32, time: 0.070, data_time: 0.017, memory: 987, student.loss: 0.1428, distiller.distance_wise_loss.0: 0.0574, distiller.angle_wise_loss.0: 0.1458, loss: 0.3460 2022-05-06 14:46:30,273 - mmcls - INFO - Saving checkpoint at 98 epochs 2022-05-06 14:46:35,656 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:46:35,656 - mmcls - INFO - Epoch(val) [98][51] accuracy_top-1: 96.3148 2022-05-06 14:46:45,825 - mmcls - INFO - Epoch [99][100/201] lr: 1.000e-04, eta: 0:00:24, time: 0.092, data_time: 0.042, memory: 987, student.loss: 0.1394, distiller.distance_wise_loss.0: 0.0562, distiller.angle_wise_loss.0: 0.1438, loss: 0.3394 2022-05-06 14:46:52,519 - mmcls - INFO - Epoch [99][200/201] lr: 1.000e-04, eta: 0:00:16, time: 0.067, data_time: 0.019, memory: 987, student.loss: 0.1366, distiller.distance_wise_loss.0: 0.0574, distiller.angle_wise_loss.0: 0.1473, loss: 0.3413 2022-05-06 14:46:52,900 - mmcls - INFO - Saving checkpoint at 99 epochs 2022-05-06 14:46:59,976 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:46:59,976 - mmcls - INFO - Epoch(val) [99][51] accuracy_top-1: 96.3148 2022-05-06 14:47:09,865 - mmcls - INFO - Epoch [100][100/201] lr: 1.000e-04, eta: 0:00:08, time: 0.089, data_time: 0.040, memory: 987, student.loss: 0.1322, distiller.distance_wise_loss.0: 0.0578, distiller.angle_wise_loss.0: 0.1469, loss: 0.3369 2022-05-06 14:47:17,032 - mmcls - INFO - Epoch [100][200/201] lr: 1.000e-04, eta: 0:00:00, time: 0.072, data_time: 0.020, memory: 987, student.loss: 0.1503, distiller.distance_wise_loss.0: 0.0571, distiller.angle_wise_loss.0: 0.1451, loss: 0.3525 2022-05-06 14:47:18,753 - mmcls - INFO - Saving checkpoint at 100 epochs 2022-05-06 14:47:24,393 - mmcls - INFO - Exp name: rkd_neck_resnet34_resnet18_8xb32_in1k.py 2022-05-06 14:47:24,394 - mmcls - INFO - Epoch(val) [100][51] accuracy_top-1: 96.1274

[humu789]

Got it，we'll check it as soon.

[humu789]

@JennieDD The size of distilled ckpt is bigger than original ckpt is due to that both of teacher's and student's ckpts are saved in one, we will fix it as soon.

[Audrey528]

I use cwd distillation method provided by you guys to compress model. I use same datasets in distillation as training process. I have three questions.

1. mAP@.5 of teacher model is 0.835, while mAP@.5 of distillation model is 0.845, is the phenomenon sound?
2. I use "scp -r"command to copy model to another server, the size of model becomes 410MB from 392MB,I am confused.
3. How can I get FPS of distillation model?

[pppppM]

The final performance may be related to the model and data you use, but I have also encountered similar experimental phenomena as you.

Question 2 may require more information so that we can help you solve it.

As for question 3, we are developing related analysis tools, but it will take some time. Currently, you can try to migrate the analysis tools from other repo

[xuhao-anhe]

在修改了配置文件训练自己的数据后，我得到了teacher的权重文件，运行的代码为：

python tools/mmcls/train_mmcls.py   configs/distill/rkd/rkd_neck_resnet34_resnet18_8xb32_in1k.py   \
--work-dir /data/dataset_81/jd_result/distill_mmcls   \
--cfg-options algorithm.distiller.teacher.init_cfg.type=Pretrained

生成的结果为：

但是出现了一个问题： 生成的经过distill 的权重文件的大小是167MB，但是teacher的权重文件只有163MB，没有起到模型瘦身的效果，这是为什么呢

你好，请问你用CWD算法蒸馏，最后的轻量化模型准确率有没有上升

[anmyles]

请问一下权重这个问题解决了吗