search

Linzaer / Ultra-Light-Fast-Generic-Face-Detector-1MB

💎1MB lightweight face detection model (1MB轻量级人脸检测模型)
MIT License
7.17k stars 1.54k forks source link

overfitting when training with my own datasets #243

Open gedance opened 3 years ago

gedance commented 3 years ago

Hello,

I've been facing overfitting issues when I train with my own datasets. I'm doing some researches for detecting vehicle plates, and there are over 60K images of vehicles, no problem with annotations. There's only one vehicle in each image, rarely two.

My issues are

  1. overfitting happens as learning rate decrease as attached

    RFB setting --milestones 150, 220 --batch-size 24 --input size 320

    running cuda

    lr_rate: 0.01 decrease at epoch 150 to 0.001, then 0.0001 at epoch 220

    I've tried adding dropouts, using Adam optimizer, adding augmentations, but didn't help. does anyone know what else I can try? Early stopping is the only answer in my case?

overfitting_img

  1. when adding more than 40K images, train does not proceed after epoch 0. I'm not sure if there's any limit but whenever i use more than 40K images as datasets training does not begin and hangs after epoch 0. Also, same thing happens when I set image size as 640 for under 40K datasets.

I am a new to ML, your experienced advice would be very appreciated.

Thanks!

kimtaehyeong commented 3 years ago

I will answer my personal thoughts for similar reasons. My guess might be that it is a model that is too light for the problem to be solved. (Actually, I can't find any other classes other than faces because of a similar problem) If you are using a pretrained model, why not try sgd, not Adam?

AryaAftab commented 1 year ago

Hi @gedance, Can you help me train the model on a custom dataset (data preparation from YOLO format to train format)? I encountered several different errors and could not resolve them.