cannot overfit to a small subset

[gorkemguzeler]

Search before asking

• [X] I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Hi,

I am working on modifications to the original backbone. To understand whether modified model works well (bug free), I try to overfit on small subset. When I was not able to overfit my modified model, i switched to trying original model (ultralytics/cfg/models/v8/yolov8.yaml) on a small dataset. Surprisingly, this model also did not overfit. Information on my small dataset: it is from MS COCO, only 10 images are present on both train and val. they are exactly same images.

I thought, since the training pipeline uses mosaic maybe this prevents me overfitting; so i updated mosaic as 0. Also other augmentations could be confusing the model, and I tried to set them to 0 too. But what else could be blocking me?

Just to be clear; everything works fine with the original dataset. I am not using a pre-trained network.

Thanks! Best regards,

Additional

No response

[glenn-jocher]

@gorkemguzeler hi there,

Overfitting a small dataset is indeed a common strategy to verify that a model has the capacity to learn. If you're unable to overfit the YOLOv8 model on a small subset of COCO, consider the following:

1. Learning Rate: Ensure the learning rate isn't too low, as this could slow down or prevent convergence.
2. Batch Size: A smaller batch size can sometimes help with overfitting on a tiny dataset.
3. Complexity: Verify that the model's complexity is appropriate for the task. A model that's too simple may not have the capacity to overfit.
4. Regularization: Disable any weight decay or other regularization techniques that might be preventing overfitting.
5. Epochs: Increase the number of epochs since overfitting might take longer with a complex model like YOLOv8.

Remember, the goal here is to intentionally overfit, so typical best practices for generalization don't apply. If you've already adjusted augmentations and are not using a pre-trained network, the above suggestions are your next steps to investigate.

[gorkemguzeler]

Hi @glenn-jocher, thanks for the suggestions, it helped a lot. To be helpful to others having a similar problem, I was able to overfit after:

• decreasing the batch size to 4,
• training for 500 epochs,
• opting Adam as optimizer with lr= 0.001 (I was trying 0.01 initially which was too high)
• obviously without data augmentation and regularization

Best,

[glenn-jocher]

Hi @gorkemguzeler,

Great to hear that you've successfully managed to overfit your model! 👍 Your adjustments are spot-on for this specific task. Thanks for sharing your solution with the community; it's sure to be helpful for others in similar situations.