Closed aslotte closed 4 years ago
Selecting the same file using the Model Builder causes Visual Studio 2019 to crash as well. Appears to be an issue reading the .csv
file, although Excel is able to parse it.
Same issue occurs when loading this dataset: https://data.world/promptcloud/product-details-on-flipkart-com
I first thought it had something to do with commas in the columns, but I've since then managed to eliminate that as a cause. The issue seems to be narrowed down to the description
column. If that column is removed, the dataset is loaded correctly. Could it potentially be the line breaks in that column that's causing the issue?
This is the error I see in the event logs. I'm running this on another computer than the one I used when seeing this issue for the first time (thus the Framework version). I would assume to see the same error on my other computer though.
Application: devenv.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: Microsoft.ML.AutoML.InferenceException
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(Microsoft.ML.MLContext, Microsoft.ML.AutoML.TextFileSample, System.Nullable`1<Char>, System.Nullable`1<Boolean>, System.Nullable`1<Boolean>)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(Microsoft.ML.MLContext, System.String, UInt32, Boolean, System.Nullable`1<Char>, System.Nullable`1<Boolean>, System.Nullable`1<Boolean>, Boolean, Boolean)
at Microsoft.ML.ModelBuilder.DataSources.FileDataSource.GetCorrectDelimiter(System.String)
at Microsoft.ML.ModelBuilder.DataSources.FileDataSource.GetListOfColumns(System.String)
at Microsoft.ML.ModelBuilder.ToolWindows.TrainModelDataContext.GetTotalFileColumn()
at Microsoft.ML.ModelBuilder.TrainModelToolWindowControl.GetDataLoadDimensions()
at Microsoft.ML.ModelBuilder.TrainModelToolWindowControl.SelectFileButton_Click(System.Object, System.Windows.RoutedEventArgs)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(System.Object, System.Windows.RoutedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(System.Object, System.Windows.RoutedEventArgs, Boolean)
at System.Windows.UIElement.RaiseEventImpl(System.Windows.DependencyObject, System.Windows.RoutedEventArgs)
at System.Windows.UIElement.RaiseEvent(System.Windows.RoutedEventArgs)
at System.Windows.Controls.Primitives.ButtonBase.OnClick()
at System.Windows.Controls.Button.OnClick()
at System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(System.Windows.Input.MouseButtonEventArgs)
at System.Windows.UIElement.OnMouseLeftButtonUpThunk(System.Object, System.Windows.Input.MouseButtonEventArgs)
at System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(System.Delegate, System.Object)
at System.Windows.RoutedEventArgs.InvokeHandler(System.Delegate, System.Object)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(System.Object, System.Windows.RoutedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(System.Object, System.Windows.RoutedEventArgs, Boolean)
at System.Windows.UIElement.ReRaiseEventAs(System.Windows.DependencyObject, System.Windows.RoutedEventArgs, System.Windows.RoutedEvent)
at System.Windows.UIElement.OnMouseUpThunk(System.Object, System.Windows.Input.MouseButtonEventArgs)
at System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(System.Delegate, System.Object)
at System.Windows.RoutedEventArgs.InvokeHandler(System.Delegate, System.Object)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(System.Object, System.Windows.RoutedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(System.Object, System.Windows.RoutedEventArgs, Boolean)
at System.Windows.UIElement.RaiseEventImpl(System.Windows.DependencyObject, System.Windows.RoutedEventArgs)
at System.Windows.UIElement.RaiseTrustedEvent(System.Windows.RoutedEventArgs)
at System.Windows.UIElement.RaiseEvent(System.Windows.RoutedEventArgs, Boolean)
at System.Windows.Input.InputManager.ProcessStagingArea()
at System.Windows.Input.InputManager.ProcessInput(System.Windows.Input.InputEventArgs)
at System.Windows.Input.InputProviderSite.ReportInput(System.Windows.Input.InputReport)
at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr, System.Windows.Input.InputMode, Int32, System.Windows.Input.RawMouseActions, Int32, Int32, Int32)
at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr, MS.Internal.Interop.WindowMessage, IntPtr, IntPtr, Boolean ByRef)
at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
at MS.Win32.HwndWrapper.WndProc(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(System.Object)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority, System.TimeSpan, System.Delegate, System.Object, Int32)
at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr, Int32, IntPtr, IntPtr)
If I modify the description
column in Excel by doing the following, it works
" "
This is obviously not a long term solution, but maybe we at least would like to include similar cleaning of non-printable chars when inferring columns?
@najeeb-kazmi Do you know if this is expected behavior?
With the "product-details-on-flipkart-com" dataset @aslotte linked, I obtain the following Model Builder Error:
Hey @aslotte , the first dateset you mentioned (https://data.world/promptcloud/fashion-products-on-amazon-com/workspace/file?filename=amazon_co-ecommerce_sample.csv) loads fine on ML.NET Model Builder on my end. However, I am also having issues with the second dataset. In addition, I am not able to import the flipkart dataset even after removing the "description" column as you have suggested.
Hi @aslotte , I believe I figured out the problem here. The Flipkart dataset you linked has empty values for certain fields, such as in line 13 for "retail price" and "discounted price" columns, which results in an error when a field that may be expecting a Boolean or an int instead sees an empty string.
The exception you have been receiving ("Exception Info: Microsoft.ML.AutoML.InferenceException") occurs when, according to InferenceException.cs, "... when AutoML is inferring the data type of a column." This means that AutoML was having issues deciding what type (int, string, boolean, etc..) a given variable was.
In the code I've written in TestCSVLoad.cs in this branch of mine, I manually load the flipkart dataset you mentioned. This is the class I've made for my IDataView:
Currently, all of the data types here are strings. When I change the types of variables that should obviously be ints (such as RetailPrice and DiscountedPrice) or boolean (IsFkAdvantagedProduct), I obtain the following error:
I believe that the error you are having and the exception I get are due to the same cause, which is misaligned columns for LoadFromTextFile.
System information
Issue
Source code / logs
Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.