How to train efficientnet on CIFAR-10 or CIFAR-100? Image size is 32x32.

[ypw-rich]

The default model input size is 224~600. What adjustments should I make to fit CIFAR-10's 32x32?

def efficientnet_params(model_name):
    """ Map EfficientNet model name to parameter coefficients. """
    params_dict = {
        # Coefficients:   width,depth,res,dropout
        'efficientnet-b0': (1.0, 1.0, 224, 0.2),
        'efficientnet-b1': (1.0, 1.1, 240, 0.2),
        'efficientnet-b2': (1.1, 1.2, 260, 0.3),
        'efficientnet-b3': (1.2, 1.4, 300, 0.3),
        'efficientnet-b4': (1.4, 1.8, 380, 0.4),
        'efficientnet-b5': (1.6, 2.2, 456, 0.4),
        'efficientnet-b6': (1.8, 2.6, 528, 0.5),
        'efficientnet-b7': (2.0, 3.1, 600, 0.5),
    }
    return params_dict[model_name]

https://github.com/lukemelas/EfficientNet-PyTorch/blob/master/efficientnet_pytorch/utils.py#L101-L114

[M0reDr1nk]

Maybe we can resize the image to 224x224 or we should adjust the step of Conv/Pooling in the model.

[lukemelas]

EfficientNet is really designed to be used on images of a specific size, but you can just take the model and apply it (probably without any modifications) to images of other sizes. Its performance should still be pretty good.

[Catosine]

EfficientNet is really designed to be used on images of a specific size, but you can just take the model and apply it (probably without any modifications) to images of other sizes. Its performance should still be pretty good.

Yeah, at least it works with 112x112 pics.

[lukemelas]

Good to hear. Let me know if this issue should be closed or if you would like it to remain open.

[ypw-rich]

Maybe modifying the code below to fit 32x32 is a better solution, because this allows the model to be trained faster.

    blocks_args = [
        'r1_k3_s11_e1_i32_o16_se0.25', 'r2_k3_s22_e6_i16_o24_se0.25',
        'r2_k5_s22_e6_i24_o40_se0.25', 'r3_k3_s22_e6_i40_o80_se0.25',
        'r3_k5_s11_e6_i80_o112_se0.25', 'r4_k5_s22_e6_i112_o192_se0.25',
        'r1_k3_s11_e6_i192_o320_se0.25',
    ]

https://github.com/lukemelas/EfficientNet-PyTorch/blob/master/efficientnet_pytorch/utils.py#L238-L243

Reducing the use of s22 allows the model to fit the 32x32 image size, but how to modify it is a problem.

[dongzhen123]

I think it is a good idea to use the pretrained model on imagenet to transfer to cifar10,and the imput size should also be modified to 224.

[ypw-rich]

The author of EfficientNet said he uses the same size as ImageNet:

https://github.com/tensorflow/tpu/issues/421#issuecomment-507365874

[daixiangzi]

today i test cifar-10(raw image is 32*32) 1.use efficientnet-b3 input_size is 300,but acc is very low 2.use efficientnet-b3 input_size is modified 32,the result is normal on conclusion,need model input_size to fit raw image size??

[HoracceFeng]

Hi guys, any updates about this issue? Recently I plan to train efficient net on my own dataset. The image size is 36*36. I have tested it by using the pretrained b0 and b4 and find out the code can run successfully. Not sure if I can change nothing and just do the transfer learning. Any ideas?

Thx.

[HoracceFeng]

Just an update: change image size to any nums seems OK since the maxpool and pad will do the work. I finally use 32*2=64 as input. Everything go smooth. I think this issue can be closed.

[ruifengma]

EfficientNet is really designed to be used on images of a specific size, but you can just take the model and apply it (probably without any modifications) to images of other sizes. Its performance should still be pretty good.

May I ask that, whatever size of image could be directly feed into the network or it's need to be resize firstly to the default pre-trained image-size(e.g. 224*224)?

[Erotemic]

I was attempting to reproduce recent EfficientNet results using the 32x32 patches as input, but even using pretrained imagenet weights my model only reaches 87.13% accuracy using the b0 model (note that this was better than the b1, b3, and b7 model).

However, when I do a training run where I resize the images to 224x224 I get much worse results. (I'm currently at 25% accuracy at epoch 69). Granted I only have 1 GPU, so I can't run with the batch size mentioned in the paper, but I'm really surprised at how difficult it is to get good results with EfficientNet, especially when training from scratch.

Has anyone been able to do any better than this on CIFAR? With a ResNet50, I can get 94-96% accuracy without too much trouble, but I can't seem to find the right hyperparameters to train efficientnet well (which is a true shame because it runs so freaking fast).

For reference the code I'm using to train CIFAR is here: https://gitlab.kitware.com/computer-vision/netharn/-/blob/dev/0.5.5/netharn/examples/cifar.py The CLI that reproduces my result is:

    python -m netharn.examples.cifar --xpu=0 --nice=efficientnet0_newaug_b128 --batch_size=128 --arch=efficientnet-b0 --optim=sgd --schedule=step-150-250 --lr=0.1 --init=kaiming_normal --augment=simple

[joaopauloschuler]

I created a program to resize the CIFAR-10 dataset:

There is a half baked example with EfficientNetB0 using the above dataset: https://www.kaggle.com/joaopauloschuler/cifar-10-128x128-resized-with-efficientnetb0-gpu

[siarez]

@Erotemic Hi! I'm exactly at the same place as you were. Did you figure it out? I really want to get it to be at least as good as ResNet50 on Cifar10, Cifar100, and Tiny ImageNet. So far no joy.