MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.28k stars 21.47k forks source link

Fix Object Detection Model Evaluation #91579

Closed KazuOnuki closed 2 years ago

KazuOnuki commented 2 years ago

In this classification model doc, it explains that custom vision services are basically conducting k-fold cross validation internally.

On the other hand, object detection model doc did not mention that the k-fold cross validation would be carried out with a custom vision.

In the object detection model, the customer may misunderstand that they do not perform k-fold cross validation on object-detection model, so we would appreciate it if you could correct the description just in case.


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

GiftA-MSFT commented 2 years ago

@KazuOnuki we'll review your feedback and get back to you shortly. Thanks.

GiftA-MSFT commented 2 years ago

@PatrickFarley can you please clarify the process used in object detection model for estimating performance? Thanks.

KazuOnuki commented 2 years ago

@GiftA-MSFT I have just asked the development department about the how to train customo-vision model, and they have given me feedback that they have changed from the k-fold-cross-validation method to a simple training-validation method around 2020, and that they are splitting the training : verification = 4 : 1 ratio. Therefore, would it be possible to add this information to the report?

PatrickFarley commented 2 years ago

@KazuOnuki Does this mean that 1/4 of the training data is used for verifying the model? Can you provide more info about how the score is calculated? I can remove the note about k-fold cross validation

PatrickFarley commented 2 years ago

I've removed the mentions of k-fold

please-close