Text classification: The output results have problem for the generated Console app and Web Api projects.

[v-Hailishi]

System Information (please complete the following information): Windows OS: Windows-11-Enterprise-21H2 ML.Net Model Builder 2022: 16.14.0.2255803 (Main Build) Microsoft Visual Studio Enterprise: 2022(17.3.6) .Net: 6.0

Describe the bug

• On which step of the process did you run into an issue: The output results have problem for the generated Console app and Web Api projects.

TestMatrix https://testpass.blob.core.windows.net/test-pass-data/wikipedia-detox-250-line-data.tsv

To Reproduce Steps to reproduce the behavior:

1. Select Create a new project from the Visual Studio start window.
2. Choose the C# Console App (.NET Core) project template.
3. Add model builder by right click on the project.
4. Select "Text classification" scenario and data source to complete the training.
5. Click "Add to solution" button on the Consume page to generate Console App and Web Api projects.
6. Run the generated Console App and Web Api projects and check their output results.

Expected behavior The output results have no problem.

Screenshot

[zewditu]

Console App missed this code Console.WriteLine($"\n\nPredicted Sentiment: {predictionResult.PredictedLabel}\n\n");

[zewditu]

On latest main WebApi for Text- classification

WebApi for data- multiclassification

it seems that something is off for both ? @luisquintanilla

[v-Hailishi]

Verified on the latest main build:16.14.0.2256401, the bug is still repro for generated Web API project.

[luisquintanilla]

It's okay for text and data classification scenario API outputs not to match because while for text classification we expect input to always to text, it's not always the case for data classification. Moving to validation.

[v-Hailishi]

Verified on the latest main build:16.14.0.2256701, the bug is still repro for generated Web API project. for API output, the score value is incorrect. It should be like {0.83....., 0.17....} (Their sum should be 1).

[zewditu]

@v-Hailishi thanks for catching this issue , it is related to this https://github.com/dotnet/machinelearning-modelbuilder/issues/2317

[zewditu]

This is prediction result calibration or sum to 1 issue.

We fix this https://github.com/dotnet/machinelearning-modelbuilder/issues/2317 for evaluate page from Model builder temporarily , in the webApi we used predictionEnginePool.Predict()which is in Ml.Net I think this issue should be transferred to ML.net

[zewditu]

I proposed the following code update to calibrate prediction results in web app for text- classification. Only for the case of text classification Add the following method in a generated project consumption code

Pass this method as handler in Programs.cs

@luisquintanilla @LittleLittleCloud ?

[luisquintanilla]

I proposed the following code update to calibrate prediction results in web app for text- classification. Only for the case of text classification Add the following method in a generated project consumption code

Pass this method as handler in Programs.cs

@luisquintanilla @LittleLittleCloud ?

What that method does is part of the prediction, rather than creating a web app with PredictionEnginePool. I think the current method is called Predict, so I would suggest keeping that name.

[LittleLittleCloud]

In the long run, I would propose putting the softmax as an option in TextClassification trainer so the final score from TextClassification trainer will sum up to one. What you think @luisquintanilla @michaelgsharp

[luisquintanilla]

In the long run, I would propose putting the softmax as an option in TextClassification trainer so the final score from TextClassification trainer will sum up to one.

Would this be a one off or something that the other deep learning classification models (i.e. object detection, image classification) would benefit from?