ultralytics / ultralytics

NEW - YOLOv8 🚀 in PyTorch > ONNX > OpenVINO > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
25.79k stars 5.13k forks source link

cls_loss and dfl_loss suddenly spike in the last 10 epochs #14110

Open tien0717 opened 5 days ago

tien0717 commented 5 days ago

Search before asking

Question

results

When I finish training YOLOv8, the cls_loss and dfl_loss suddenly spike in the last 10 epochs. Could you please tell me where there might be an error in the code?

Additional

No response

github-actions[bot] commented 5 days ago

👋 Hello @tien0717, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Status

Ultralytics CI

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Y-T-G commented 5 days ago

Last 10 epochs run without mosaic augmentation. It's expected.

glenn-jocher commented 5 days ago

@Y-T-G hello,

Thank you for reaching out. The spike in cls_loss and dfl_loss during the last 10 epochs is likely due to the model transitioning from training with mosaic augmentation to training without it. This is a standard practice to stabilize the model's learning and improve its performance on more realistic, non-augmented data.

To provide a more accurate diagnosis, could you please share a minimum reproducible example of your training script? This will help us better understand the issue and offer more targeted advice. You can find guidelines for creating a reproducible example here.

Additionally, please ensure you are using the latest version of the Ultralytics YOLO package, as updates often include important bug fixes and improvements.

Feel free to share any further details or questions you might have. We're here to help!