search

dotnet / machinelearning-modelbuilder

Simple UI tool to build custom machine learning models.
Creative Commons Attribution 4.0 International
264 stars 56 forks source link

Specified Label '<LabelName>' was not found #1027

Closed roberthleone closed 3 years ago

roberthleone commented 3 years ago

Operating System: windows 10 pro

Provide details about the problem you're experiencing. Include your operating system version, exact error message, code sample, and anything else that is relevant.

I created a dotnet core console app, right clicked the console app project, added machine learning, chose a scenario, selected a csv file for training which had 7 columns, selected the column I wanted it to predict, selected 5 of them to use as part of the learning, then I get the error listed above I used the as a substitute for my actual label name because I didn't think it really mattered.

beccamc commented 3 years ago

@roberthleone Sorry you ran into this problem! Are you able to share you dataset (or part of your dataset) with us?

roberthleone commented 3 years ago

DateAndNumbers.csv.txt

roberthleone commented 3 years ago

I added the .txt at the end because I wasn't able to add the .csv file type to this report not to the app in visual studio 2019 community v16.7.5

roberthleone commented 3 years ago

@beccamc, did the file supplied help figure out the root cause of my issue?

beccamc commented 3 years ago

@roberthleone Thanks for supplying your data! I wasn't able to reproduce, the model trained successfully for me. Is your VS in a different language than English?

image

sfiruch commented 3 years ago

I hit the same problem. Here's the stack trace

Specified label column 'Test' was not found.
   at Microsoft.ML.AutoML.ColumnTypeInference.GetAndValidateLabelColumn(Arguments args, IntermediateColumn[] cols)
   at Microsoft.ML.AutoML.ColumnTypeInference.InferTextFileColumnTypesCore(MLContext context, IMultiStreamSource fileSource, Arguments args)
   at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumnTypes(MLContext context, TextFileSample sample, ColumnSplitResult splitInference, Boolean hasHeader, Nullable`1 labelColumnIndex, String label)
   at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
   at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParameter config) in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 271
   at Microsoft.ML.ModelBuilder.AutoMLEngine.<StartTrainingAsync>d__26.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 114

How to repro:

  1. Create a .NET Core console app
  2. Add "Machine Learning"
  3. Select "Custom scenario"
  4. Load the above example file
  5. Pick "Test" as column to predict
  6. Click "Train"
  7. Click "Start Training"

Things I noticed

beccamc commented 3 years ago

Thanks for reporting @deiruch! What version of Model Builder are you on? Based on your report I think you're on an older version. In our most recent release (16.3.0.2056001) we've completely changed how your data file is processed. Could you give it another try with the new build?

sfiruch commented 3 years ago

This happens with ML.NET Model Builder (Preview) v16.3.0.2056001.

beccamc commented 3 years ago

Thanks! I'll take a look

beccamc commented 3 years ago

@deiruch I'm able to train successfully with the dataset provided by roberthleone. Are you using the same dataset?

I also used an English VS and English Windows.

  1. Are your time fields also formatted like this? 3/20/2020
  2. Do you maybe have a trailing comma at the end of every line in your dataset?
sfiruch commented 3 years ago

I tested with the file roberthleone provided above, without modifications. So:

  1. Yes, the dates in the file are formatted like 3/20/2020.
  2. No
csdahlberg commented 3 years ago

I have encountered the same error when following the step-by-step instructions from https://dotnet.microsoft.com/learn/ml-dotnet/get-started-tutorial/intro, including using the sample data file of https://raw.githubusercontent.com/dotnet/machinelearning/master/test/data/wikipedia-detox-250-line-data.tsv.

I see the same behavior with versions 16.1.0.2027905 and 16.3.0.2056001 of the ML.NET Model Builder (Preview) extension, and in both 16.8.2 and 16.9.0 Preview 2.0 of Visual Studio 2019 Community Edition. It does not seem to matter which scenario or training task I select.

bsstahl commented 3 years ago

Is there any update here? A workaround maybe? I'm running into this same problem with the same 2 versions of the tool (16.1.0.20207905 then upgraded to 16.3.0.2056001). It is turning what should have been a very quick experiment to see if I could get any predictive value out of a dataset into a frustratingly difficult exercise. I have tried the data in both csv and tsv format, tried smaller sets, tried adding and eliminating any trailing lines. I can't share the exact dataset but I could make one that would demonstrate the problem if it will help.

beccamc commented 3 years ago

@bsstahl Yes, please share a dataset that repros the problem! Can you also please share your machine and VS language settings?

sfiruch commented 3 years ago

@bsstahl Yes, please share a dataset that repros the problem! Can you also please share your machine and VS language settings?

As I posted in November already, I can repro the problem with the OP's dataset. Let me know if you need additional information from me.

bsstahl commented 3 years ago

I have what I think is some good news. It turns out that one of my machines works with this dataset, while another doesn't. I've posted all of the details I can think of at this point documenting the differences between these two setups as well as the dataset I used. Hopefully that will lead to a resolution.

The key thing to me is that the UIs look very different. So different in fact, that it doesn't seem possible that I am actually running the same version of the tooling on both machines. However, for the things I know to look at, the versions are the same. Perhaps there is something else that is installed that I need to check?

You can find my write-up here: https://cognitiveinheritance.com/Posts/Troubleshooting-Machine-Learning-ModelBuilder-Issue-1027.html

beccamc commented 3 years ago

@bsstahl Thank you for the details!! The UI from the machine having the failure is very old. Did you have a version of Model Builder installed on the problem machine prior to when we started shipping with VS? This would have been sometime last summer.

@deiruch @csdahlberg Are both of you seeing a UI similar to this? image

If the UI looks like this for all of you, the problem is that the extension upgrade is broken. Even though the extension manager says 16.3.0.2056001 you're actually on a much much older version. This would explain why I can't get a repro of the problem no matter what machine settings I change.

sfiruch commented 3 years ago

I reinstalled VS and switched to VS Preview last week (unrelated problem). I can no longer repro the problem - it works as expected. 👍

beccamc commented 3 years ago

@deiruch Excellent! So happy that worked.

To give more context on the situation. We believe the problem is there are two version installed. We shipped one version on a VS Preview channel last summer. We then changed how our extension is packaged/shipped. The old packaged version and the new packaged version have the same package ID, but based on breaking changes the new version didn't overwrite the old. Both versions are still there, but the new one is showing in the extension manager and the old one is loading when you try to use the extension.

To fix: Please uninstall Model Builder - both from the Extension Manager Window and from the Visual Studio Installer. Then re-install from the VS Installer, and update to the latest version from the Marketplace. You should see a new UI similar to this...

image

Sorry about the delay in resolving this issue everyone! Thank you for your patience, and let me know if complete Model Builder uninstall/reinstall doesn't work. If the old package is still sticking around you may need to reinstall the VS Preview build.

bsstahl commented 3 years ago

I was unable to fully uninstall the Extension. Once I rolled-back (and restarted) to the point where the Extension manager offered me the opportunity to uninstall, it would open the VS Installer and fail with the message Microsoft.ML.ModelBuilder: No matching workload or component found. I ended up having to do a full uninstall-reinstall of Visual Studio. That worked, and I can now generate models on that machine using this tool. Thanks @beccamc and everyone else who worked to figure this out.

beccamc commented 3 years ago

Glad it worked! I'll also re-try my uninstall instructions and see if it needs to be done in the other order. Thanks for the heads up.

beccamc commented 3 years ago

Has anyone else been able to fix their problem with the above advice?

beccamc commented 3 years ago

It seems like this is solved for those who tried the solution. If anyone else runs into problems, and the above advice doesn't work, please @ mention me for help. Thank you everyone for reporting and helping to resolve this!