search

ultralytics / ultralytics

Ultralytics YOLO11 🚀
https://docs.ultralytics.com
GNU Affero General Public License v3.0
32.36k stars 6.22k forks source link

How to Calculate Accuracy in Practical Applications？ #15720

Closed shengyu27 closed 2 days ago

shengyu27 commented 2 months ago

Search before asking

Question

For example, I am currently training a YOLO model for detecting safety helmets, and I need to call the camera for detection. For example, I need to test the accuracy of this model in field applications, which is different from the test set. The test set has labeled real values, but the application does not. So, how can I easily and quickly judge the quality of the trained model?

Additional

No response

glenn-jocher commented 2 months ago

@shengyu27 to assess the accuracy of your YOLO model in practical applications without labeled data, you can use metrics like precision, recall, and F1 score during validation on a labeled test set. For real-time applications, consider using a subset of manually verified predictions to estimate performance. For more details, please refer to our Model Testing Guide.

shengyu27 commented 2 months ago

What you are actually saying is to manually screen and analyze after testing, and finally obtain accuracy or precision. Is there no automatic detection of accuracy within a specific time frame other than manual? I actually know that theoretically, annotation is necessary to calculate.

glenn-jocher commented 2 months ago

@shengyu27 to automatically assess model accuracy in real-time applications without annotations, consider using a subset of manually verified predictions or leveraging semi-supervised learning techniques. This can provide an estimate of performance without full manual annotation.

github-actions[bot] commented 1 week ago

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐