search

dotnet / machinelearning-modelbuilder

Simple UI tool to build custom machine learning models.
Creative Commons Attribution 4.0 International
263 stars 56 forks source link

Top 5 models explored summary is not correct. It doesn't match to the best model accuracy #10

Closed rustd closed 5 years ago

rustd commented 5 years ago

We are not showing the top 5 models correctly. Best model accuracy shown in the summary doesn't match the one in Top 5 models explored.

mlnet auto-train -d "C:\builddemo\buildcustomerreviews.tsv" -T binary-classification -n Sentiment -x 60 Dataset attached(rename to .tsv)

buildcustomerreviews.txt

image

rustd commented 5 years ago

Another user reported a similar bug https://github.com/dotnet/machinelearning-samples/issues/442

tbombach commented 5 years ago

This is a display issue. We keep track of all the models that have been output during training. When we're displaying them, I thought we wanted to group by trainer name in the top 5 models table. I grouped by trainer name before sorting by accuracy, which led to the visual bug.

However, looking at the CLI output, we want to display multiple iterations with the same algorithm if they end up in the top 5. So I'll remove the group by so that the UI table is the same as the CLI output. This will fix the issue because the most accurate model is the one we will use for both the label in the Output section and the Top 5 models explored table.

rustd commented 5 years ago

https://github.com/dotnet/machinelearning/issues/3734