LoadRawImageBytes does not load anything

[atkinsonbg]

System Information (please complete the following information):

• OS & Version: [e.g. Windows 10]
  • Tested on MacOS 11.6 Big Sur & Windows 11
• ML.NET Version: [e.g. ML.NET v1.5.5]
  • ML.NET v1.7.0
• .NET Version: [e.g. .NET 5.0]
  • .NET 6.0

Describe the bug Following the tutorial list here, (https://docs.microsoft.com/en-us/samples/dotnet/machinelearning-samples/mlnet-image-classification-transfer-learning/) and the LoadRawImageBytes does not appear to load images from the directory.

To Reproduce Steps to reproduce the behavior:

Code used:

IEnumerable<ImageData> images = LoadImagesFromCsv("Data/training_data.csv");

            IDataView imageData = mlContext.Data.LoadFromEnumerable(images);
            IDataView shuffledData = mlContext.Data.ShuffleRows(imageData, shuffleSource: true);

            var previewshuffledData = shuffledData.Preview();

            var preprocessingPipeline = mlContext.Transforms.Conversion
                                                .MapValueToKey(
                                                    inputColumnName: "Label",
                                                    outputColumnName: "LabelAsKey")
                                                .Append(mlContext.Transforms.LoadRawImageBytes(
                                                    outputColumnName: "Image",
                                                    imageFolder: "Data/LocalImages",
                                                    inputColumnName: "ImagePath"));

            IDataView preProcessedData = preprocessingPipeline
                    .Fit(shuffledData)
                    .Transform(shuffledData);

            var previewPreprocessedData = preProcessedData.Preview();

previewshuffledData:

previewPreprocessedData:

Expected behavior Expecting to see a RowView after LoadRawImageBytes of 63, which matches the number of images in the dataset.

Screenshots, Code, Sample Projects If applicable, add screenshots, code snippets, or sample projects to help explain your problem.

Additional context Add any other context about the problem here.

[luisquintanilla]

@atkinsonbg instead of using the Preview method, can you try converting the IDataView to an IEnumerable and iterate over that?

var previewPreprocessedData = mlContext.Data.CreateEnumerable(preProcessedData, reuseRowObject:false);

[atkinsonbg]

@luisquintanilla I change it to: var previewPreprocessedData = mlContext.Data.CreateEnumerable<ImagePredictionInput>(preProcessedData, reuseRowObject: false);

Result

:

[atkinsonbg]

I pulled the sample code from here: https://github.com/dotnet/machinelearning-samples/tree/main/samples/csharp/getting-started/DeepLearning_ImageClassification_Binary

And that is finding the images just fine:

But this code is so simple, I'm not sure where the breakdown is happening.

[luisquintanilla]

@atkinsonbg if you replace the images in the assets directory of the code sample with your own images, does it work?

[atkinsonbg]

@luisquintanilla I am so sorry, I have wasted your time today. I found the culprit after running your code, it was in the CSVHelper package found here: https://www.nuget.org/packages/CsvHelper/

Stripping that out and manually creating some input classes:

public static IEnumerable<ImageData> LoadImagesFromCsv(string csvPath)
        {
            //TextReader reader = new StreamReader(csvPath);
            //var csvReader = new CsvReader(reader, CultureInfo.CurrentCulture);
            //return csvReader.GetRecords<ImageData>();

            List<ImageData> list = new List<ImageData>();
            list.Add(new ImageData
            {
                ImagePath = "15970_2.jpg",
                Label = "Navy"
            });
            list.Add(new ImageData
            {
                ImagePath = "39386_2.jpg",
                Label = "Blue"
            });
            return list.AsEnumerable();
        }

Results in the images being processed:

That's what I get for trying to be fancy. Will parse the CSV another way. Thank you so much for jumping in and taking a look.

[luisquintanilla]

@atkinsonbg No problem. Thanks for closing the issue.

[atkinsonbg]

Just to follow-up, I was still able to use the CSVHelper package, but needed to change the implementation to the following:

public static IEnumerable<ImageData> LoadImagesFromCsv(string csvPath)
        {
            var records = new List<ImageData>();
            using (var reader = new StreamReader(csvPath))
            using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
            {
                csv.Read();
                csv.ReadHeader();
                while (csv.Read())
                {
                    var record = new ImageData
                    {
                        ImagePath = csv.GetField("ImagePath"),
                        Label = csv.GetField("Label")
                    };
                    records.Add(record);
                }
            }

            return records.AsEnumerable();
        }

    Working great now. Thanks again!