Ideas

[kktsubota]

Backbone

• EfficientNet
• https://github.com/zhanghang1989/ResNeSt

Pre-trained model

• better pre-trained model than ImageNet-1.2M pre-trained model
  • Weakly supervised Instagram-1G (tag)
  • https://github.com/Cadene/pretrained-models.pytorch

Data Augmentation

• CutMix [2] = Mixup + Cutout
  • Is this really implemented?
• AutoAugment [5] or faster model

How to use test data

• "Temporal Ensembling for Semi-Supervised Learning"
• training with noisy student [4]
• semi-supervised learning for object detection [3] instead of pseudo labeling
  • consisntency learning is a basic idea for semi-supervised learning for classification (e.g. Temporal Ensembling, Mean Teacher, MixMatch, FixMatch)

Inference

• Increase the resolution at the test time [1]: この手法自体はglobal average poolingを仮定

[1] https://papers.nips.cc/paper/9035-fixing-the-train-test-resolution-discrepancy [2] https://openaccess.thecvf.com/content_ICCV_2019/papers/Yun_CutMix_Regularization_Strategy_to_Train_Strong_Classifiers_With_Localizable_Features_ICCV_2019_paper.pdf [3] https://papers.nips.cc/paper/9259-consistency-based-semi-supervised-learning-for-object-detection.pdf [4] https://ai-scholar.tech/articles/treatise/noisy-student-ai-379 [5] https://openaccess.thecvf.com/content_CVPR_2019/html/Cubuk_AutoAugment_Learning_Augmentation_Strategies_From_Data_CVPR_2019_paper.html

[kiccho1101]

https://ai-scholar.tech/articles/treatise/CVPR2020-study コンペに直接は関係ないけど、勉強になりますなあ