Closed v-Hailishi closed 1 year ago
Console App missed this code
Console.WriteLine($"\n\nPredicted Sentiment: {predictionResult.PredictedLabel}\n\n");
On latest main WebApi for Text- classification
WebApi for data- multiclassification
it seems that something is off for both ? @luisquintanilla
Verified on the latest main build:16.14.0.2256401, the bug is still repro for generated Web API project.
It's okay for text and data classification scenario API outputs not to match because while for text classification we expect input to always to text, it's not always the case for data classification. Moving to validation.
Verified on the latest main build:16.14.0.2256701, the bug is still repro for generated Web API project. for API output, the score value is incorrect. It should be like {0.83....., 0.17....} (Their sum should be 1).
@v-Hailishi thanks for catching this issue , it is related to this https://github.com/dotnet/machinelearning-modelbuilder/issues/2317
This is prediction result calibration or sum to 1 issue.
We fix this https://github.com/dotnet/machinelearning-modelbuilder/issues/2317 for evaluate page from Model builder temporarily , in the webApi we used predictionEnginePool.Predict()
which is in Ml.Net
I think this issue should be transferred to ML.net
I proposed the following code update to calibrate prediction results in web app for text- classification. Only for the case of text classification Add the following method in a generated project consumption code
Pass this method as handler in Programs.cs
@luisquintanilla @LittleLittleCloud ?
I proposed the following code update to calibrate prediction results in web app for text- classification. Only for the case of text classification Add the following method in a generated project consumption code
Pass this method as handler in Programs.cs
@luisquintanilla @LittleLittleCloud ?
What that method does is part of the prediction, rather than creating a web app with PredictionEnginePool. I think the current method is called Predict
, so I would suggest keeping that name.
In the long run, I would propose putting the softmax as an option in TextClassification trainer so the final score from TextClassification trainer will sum up to one. What you think @luisquintanilla @michaelgsharp
In the long run, I would propose putting the softmax as an option in TextClassification trainer so the final score from TextClassification trainer will sum up to one.
Would this be a one off or something that the other deep learning classification models (i.e. object detection, image classification) would benefit from?
System Information (please complete the following information): Windows OS: Windows-11-Enterprise-21H2 ML.Net Model Builder 2022: 16.14.0.2255803 (Main Build) Microsoft Visual Studio Enterprise: 2022(17.3.6) .Net: 6.0
Describe the bug
TestMatrix https://testpass.blob.core.windows.net/test-pass-data/wikipedia-detox-250-line-data.tsv
To Reproduce Steps to reproduce the behavior:
Expected behavior The output results have no problem.
Screenshot