Is there anyone train the resattentionnet on ImageNet?
The paper didn't provide the batchsize for ImageNet training. So I set the batchsize=256/lr=0.1 which is a common setting, but the training result (top1-acc: 77.64) is much lower than paper reported (top1-acc: 78.24) ! More details about hyperparameters are listed as below. The epoch setting is converted from the iteration which is mentioned in paper. If we set the batchsize as 256, then there is 5k iteration in 1 epoch. According to the paper, we should decay the learning rate at 200k/5k=40, 400k/5k=80, 500k/5k=100 epoch, and terminate training at 530/5k=106 epoch.
The learning rate is divided by 10 at 200k, 400k, 500k iterations. We terminate training at 530k iterations.
Hyperparameter settings
args.epochs = 106
args.batch_size = 256
### data transform: RandomResizeCrop(224)/HorizontalFlip(0.5)/ChangeLight(AlexNet color augmenation)/Normalize() are used in training
args.autoaugment = False
args.colorjitter = False
args.change_light = True # standard color augmentation from AlexNet
### optimizer
args.optimizer = 'SGD'
args.lr = 0.1
args.momentum = 0.9
args.weigh_decay_apply_on_all = True # TODO: weight decay apply on which params
args.weight_decay = 1e-4
args.nesterov = True
### criterion
args.labelsmooth = 0
### lr scheduler
args.scheduler = 'uneven_multistep'
args.lr_decay_rate = 0.1
args.lr_milestone = [40, 80, 100]
Is there anyone train the resattentionnet on ImageNet?
The paper didn't provide the batchsize for ImageNet training. So I set the batchsize=256/lr=0.1 which is a common setting, but the training result (top1-acc: 77.64) is much lower than paper reported (top1-acc: 78.24) ! More details about hyperparameters are listed as below. The epoch setting is converted from the iteration which is mentioned in paper. If we set the batchsize as 256, then there is 5k iteration in 1 epoch. According to the paper, we should decay the learning rate at 200k/5k=40, 400k/5k=80, 500k/5k=100 epoch, and terminate training at 530/5k=106 epoch.
Hyperparameter settings