Predict In-Memory ImageData in Image Classification

[DirkKramer]

To classify images faster it would be helpfull to Predict on In-Memory imagedata (byte[]).

It would be very helpfull in for example my situation where i get images from a Azure Kinect camera, I now have to Write and Read from disk alll the time.

       public ModelOutput Predict(byte[] bytes)
        {
            var input = new ModelInput();
            input.ImageData = bytes;
            return PredictionEngine.Predict(input);
        }

OR

   public ModelOutput Predict(Bitmap bitmap)
        {
            var input = new ModelInput();
            input.Bitmap = bitmap;
            return PredictionEngine.Predict(input);
        }

[GreenShadeZhang]

I agree with you.

[JakeRadMSFT]

Thanks for this feedback! It is possible from an ML.NET perspective but we don't generate the code through Model Builder. Let me see if I can help provide that code.

[DirkKramer]

Thanks for this feedback! It is possible from an ML.NET perspective but we don't generate the code through Model Builder. Let me see if I can help provide that code.

That would be great, thanks in advance.

[GreenShadeZhang]

Thanks for this feedback! It is possible from an ML.NET perspective but we don't generate the code through Model Builder. Let me see if I can help provide that code.

That's great, i hope to see this feature demonstration in the sample code.

[JakeRadMSFT]

Related to #635

[JakeRadMSFT]

Input/Output Classes

public class ModelInput
        {
            [ColumnName(@"Label")]
            public string Label { get; set; }

            [ColumnName(@"ImageSource")]
            public string ImageSource { get; set; }

        }

        public class ModelInputBytes
        {
            [ColumnName(@"Label")]
            public string Label { get; set; }

            [ColumnName(@"Features")]
            public byte[] ImageBytes { get; set; }

        }

        public class ModelOutput
        {
            [ColumnName("PredictedLabel")]
            public string Prediction { get; set; }

            public float[] Score { get; set; }
        }

Consumption Code

public static ModelOutput Predict(ModelInput input)
        {
            MLContext mlContext = new MLContext();

            // Load model & create prediction engine
            ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);

            ITransformer dataPreProcessTransform = LoadImageFromFileTransformer(input, mlContext);

            var predEngine = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(dataPreProcessTransform.Append(mlModel));

            ModelOutput result = predEngine.Predict(input);
            return result;
        }

        public static ITransformer LoadImageFromFileTransformer(ModelInput input, MLContext mlContext)
        {
            var dataPreProcess = mlContext.Transforms.Conversion.MapValueToKey(@"Label", @"Label")
                                    .Append(mlContext.Transforms.LoadRawImageBytes(@"ImageSource_featurized", @"ImageSource"))
                                    .Append(mlContext.Transforms.CopyColumns(@"Features", @"ImageSource_featurized"));

            var dataView = mlContext.Data.LoadFromEnumerable(new[] { input });
            var dataPreProcessTransform = dataPreProcess.Fit(dataView);
            return dataPreProcessTransform;
        }

        public static ModelOutput PredictFromBytes(ModelInputBytes input)
        {
            MLContext mlContext = new MLContext();

            // Load model & create prediction engine
            ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);

            var predEngine = mlContext.Model.CreatePredictionEngine<ModelInputBytes, ModelOutput>(mlModel);

            ModelOutput result = predEngine.Predict(input);
            return result;
        }

Training Code

public static ITransformer RetrainPipeline(MLContext context, IDataView trainData)
        {

            var dataPreProcess = context.Transforms.Conversion.MapValueToKey(@"Label", @"Label")
                                    .Append(context.Transforms.LoadRawImageBytes(@"ImageSource_featurized", @"ImageSource"))
                                    .Append(context.Transforms.CopyColumns(@"Features", @"ImageSource_featurized"));

            var trainingPipeline = context.MulticlassClassification.Trainers.ImageClassification(labelColumnName: @"Label")
                                    .Append(context.Transforms.Conversion.MapKeyToValue(@"PredictedLabel", @"PredictedLabel"));

            var processedData = dataPreProcess.Fit(trainData).Transform(trainData);

            var model = trainingPipeline.Fit(processedData);

            return model;
        }

[JakeRadMSFT]

Essentially in the above you create a DataPreProcess pipeline and a TrainingPipeline but only save the TrainingPipeline to disk.

This way you can chose your method of DataPreProcess when you consume the model.

Option A: Load image from disk and pass to model. Option B: Just pass the byte array to the model.

[JakeRadMSFT]

Hopefully this code helps folks that are blocked! We'll be updating Model Builder to generate models and consumption code to support both methods of prediction.

[Cyrusbm]

Input/Output Classes

public class ModelInput
        {
            [ColumnName(@"Label")]
            public string Label { get; set; }

            [ColumnName(@"ImageSource")]
            public string ImageSource { get; set; }

        }

        public class ModelInputBytes
        {
            [ColumnName(@"Label")]
            public string Label { get; set; }

            [ColumnName(@"Features")]
            public byte[] ImageBytes { get; set; }

        }

        public class ModelOutput
        {
            [ColumnName("PredictedLabel")]
            public string Prediction { get; set; }

            public float[] Score { get; set; }
        }

Consumption Code

public static ModelOutput Predict(ModelInput input)
        {
            MLContext mlContext = new MLContext();

            // Load model & create prediction engine
            ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);

            ITransformer dataPreProcessTransform = LoadImageFromFileTransformer(input, mlContext);

            var predEngine = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(dataPreProcessTransform.Append(mlModel));

            ModelOutput result = predEngine.Predict(input);
            return result;
        }

        public static ITransformer LoadImageFromFileTransformer(ModelInput input, MLContext mlContext)
        {
            var dataPreProcess = mlContext.Transforms.Conversion.MapValueToKey(@"Label", @"Label")
                                    .Append(mlContext.Transforms.LoadRawImageBytes(@"ImageSource_featurized", @"ImageSource"))
                                    .Append(mlContext.Transforms.CopyColumns(@"Features", @"ImageSource_featurized"));

            var dataView = mlContext.Data.LoadFromEnumerable(new[] { input });
            var dataPreProcessTransform = dataPreProcess.Fit(dataView);
            return dataPreProcessTransform;
        }

        public static ModelOutput PredictFromBytes(ModelInputBytes input)
        {
            MLContext mlContext = new MLContext();

            // Load model & create prediction engine
            ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);

            var predEngine = mlContext.Model.CreatePredictionEngine<ModelInputBytes, ModelOutput>(mlModel);

            ModelOutput result = predEngine.Predict(input);
            return result;
        }

Training Code

public static ITransformer RetrainPipeline(MLContext context, IDataView trainData)
        {

            var dataPreProcess = context.Transforms.Conversion.MapValueToKey(@"Label", @"Label")
                                    .Append(context.Transforms.LoadRawImageBytes(@"ImageSource_featurized", @"ImageSource"))
                                    .Append(context.Transforms.CopyColumns(@"Features", @"ImageSource_featurized"));

            var trainingPipeline = context.MulticlassClassification.Trainers.ImageClassification(labelColumnName: @"Label")
                                    .Append(context.Transforms.Conversion.MapKeyToValue(@"PredictedLabel", @"PredictedLabel"));

            var processedData = dataPreProcess.Fit(trainData).Transform(trainData);

            var model = trainingPipeline.Fit(processedData);

            return model;
        }

Thanks for sharing, but this does not work for me. I'm using model trained by model builder and there is no "ImageSource_featurized" column in it.

[LittleLittleCloud]

967

[ghost]

Please can additional overload methods be added for a Uri too (not just the byte[]) please?

// Build location "C:\GitHub\Repos\MLProject\MLProject\bin\Debug\netcoreapp3.1"
var input = @"https:\azurestorage.blob.core.windows.net\container\Image.jpg";
ModelOutput result = predEngine.Predict(input);      // Additional overflow arguments for this method please....

Error: System.IO.IOException: 'The filename, directory name, or volume label syntax is incorrect. : 'C:\GitHub\Repos\MLProject\MLProject\bin\Debug\netcoreapp3.1\https:\azurestorage.blob.core.windows.net\container\Image.jpg'

I am unable to override the input as the appended string is occurring in the Predict method, which I do not have access too.

[lehong3000]

Thanks for this feedback! It is possible from an ML.NET perspective but we don't generate the code through Model Builder. Let me see if I can help provide that code.

Please add similar solution for Object Detection Model

[mturetchi]

Hopefully this code helps folks that are blocked! We'll be updating Model Builder to generate models and consumption code to support both methods of prediction.

I tried this solution but it doesn't work...

[LittleLittleCloud]

What's the error says?

[mturetchi]

What's the error says?

An error occurs because ModelInputBytes does not contain ImageSource, and if I add this field, (thinking that Features Column will be copied to ImageSource), then the result will be an array of scores with values 0

[sahgilbert]

Regarding This comment above I too have the same problem and would like to see an overload for Uri's being supported when making predictions.

This will of course allow us to upload images via an API to Azure Blob Storage, and then make predictions against these images simply by passing in the public blob container Uri.

Thank you.

[vzhuqin]

@LittleLittleCloud Could you provide me with information on how to validate this issue?

[vzhuqin]

Verified this issue on latest main: 16.9.1.2156701, finished training without error for Image classification (Local (CPU) & Azure) and Object detection scenarios, and everything work fine, details as below. Image Local: Evaluate:

Consume:

• Can add projects to solution and run (ConsoleApp & Notebook) or build (WebApi) them successfully

Image Azure: Evaluate:

Consume:

• Can add projects to solution and run (ConsoleApp & Notebook) or build (WebApi) them successfully

Object: Evaluate: Consume:

• Can add projects to solution and run (ConsoleApp & Notebook) or build (WebApi) them successfully

Code snippet: copied them as below screenshot.

[flake-neural]

Input/Output Classes

public class ModelInput
        {
            [ColumnName(@"Label")]
            public string Label { get; set; }

            [ColumnName(@"ImageSource")]
            public string ImageSource { get; set; }

        }

        public class ModelInputBytes
        {
            [ColumnName(@"Label")]
            public string Label { get; set; }

            [ColumnName(@"Features")]
            public byte[] ImageBytes { get; set; }

        }

        public class ModelOutput
        {
            [ColumnName("PredictedLabel")]
            public string Prediction { get; set; }

            public float[] Score { get; set; }
        }

Consumption Code

public static ModelOutput Predict(ModelInput input)
        {
            MLContext mlContext = new MLContext();

            // Load model & create prediction engine
            ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);

            ITransformer dataPreProcessTransform = LoadImageFromFileTransformer(input, mlContext);

            var predEngine = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(dataPreProcessTransform.Append(mlModel));

            ModelOutput result = predEngine.Predict(input);
            return result;
        }

        public static ITransformer LoadImageFromFileTransformer(ModelInput input, MLContext mlContext)
        {
            var dataPreProcess = mlContext.Transforms.Conversion.MapValueToKey(@"Label", @"Label")
                                    .Append(mlContext.Transforms.LoadRawImageBytes(@"ImageSource_featurized", @"ImageSource"))
                                    .Append(mlContext.Transforms.CopyColumns(@"Features", @"ImageSource_featurized"));

            var dataView = mlContext.Data.LoadFromEnumerable(new[] { input });
            var dataPreProcessTransform = dataPreProcess.Fit(dataView);
            return dataPreProcessTransform;
        }

        public static ModelOutput PredictFromBytes(ModelInputBytes input)
        {
            MLContext mlContext = new MLContext();

            // Load model & create prediction engine
            ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);

            var predEngine = mlContext.Model.CreatePredictionEngine<ModelInputBytes, ModelOutput>(mlModel);

            ModelOutput result = predEngine.Predict(input);
            return result;
        }

Training Code

public static ITransformer RetrainPipeline(MLContext context, IDataView trainData)
        {

            var dataPreProcess = context.Transforms.Conversion.MapValueToKey(@"Label", @"Label")
                                    .Append(context.Transforms.LoadRawImageBytes(@"ImageSource_featurized", @"ImageSource"))
                                    .Append(context.Transforms.CopyColumns(@"Features", @"ImageSource_featurized"));

            var trainingPipeline = context.MulticlassClassification.Trainers.ImageClassification(labelColumnName: @"Label")
                                    .Append(context.Transforms.Conversion.MapKeyToValue(@"PredictedLabel", @"PredictedLabel"));

            var processedData = dataPreProcess.Fit(trainData).Transform(trainData);

            var model = trainingPipeline.Fit(processedData);

            return model;
        }

Thanks for sharing, but this does not work for me. I'm using model trained by model builder and there is no "ImageSource_featurized" column in it.

I have the same issue with the solution provided above by JakeRadMSFT. Did you manage to find a fix or another solution for this? I thought there was maybe a step missing and I would have to rebuild something, however the modelbuilder will remove the custom code when I do that.

[danieldugger]

I am in desperate need of assistance with the issue being described within this ticket. I am using ML.net within VS 2022 to develop a monkeypox lesion detection system. I am able to use the ImagePath, however, I do not want to write the images to disk when received via my API.

[beccamc]

@danieldugger Which issue specifically are you seeing? Does this sample not work for you? https://github.com/dotnet/machinelearning-modelbuilder/issues/851#issuecomment-833967834

[danieldugger]

@danieldugger Which issue specifically are you seeing? Does this sample not work for you? #851 (comment)

Thanks for the quick reply. I am getting the following: System.ArgumentOutOfRangeException: 'Could not find input column 'ImageSource' (Parameter 'inputSchema')'

[danieldugger]

@danieldugger Which issue specifically are you seeing? Does this sample not work for you? #851 (comment)

Thanks for the quick reply. I am getting the following: System.ArgumentOutOfRangeException: 'Could not find input column 'ImageSource' (Parameter 'inputSchema')'

namespace Monkeypox_ConsoleApp { class Program { static void Main(string[] args) { // Create single instance of sample data from first line of dataset for model input Monkeypox.ModelInput sampleData = new Monkeypox.ModelInput() { ImageSource = @"C:\Users\ddugger\Downloads\MPX Pictures\ML.net\Positive\M01_01.jpg", };

        Monkeypox.ModelInputBytes modelInputBytes = new Monkeypox.ModelInputBytes()
        {
            Label="Positive",
            ImageBytes=System.IO.File.ReadAllBytes(@"C:\Users\ddugger\Downloads\MPX Pictures\ML.net\Positive\M01_01.jpg")
        };

        // Make a single prediction on the sample data and print results
        var predictionResult = Monkeypox.Predict(sampleData);
        predictionResult = Monkeypox.PredictFromBytes(modelInputBytes);
        Console.WriteLine("Using model to make single prediction -- Comparing actual Label with predicted Label from sample data...\n\n");

        Console.WriteLine($"ImageSource: {@"C:\Users\ddugger\Downloads\MPX Pictures\ML.net\Negative\NM01_01.jpg"}");

        Console.WriteLine($"\n\nPredicted Label value: {predictionResult.Prediction} \nPredicted Label scores: [{String.Join(",", predictionResult.Score)}]\n\n");
        Console.WriteLine("=============== End of process, hit any key to finish ===============");
        Console.ReadKey();
    }
}

}

[LittleLittleCloud]

@danieldugger Can you also share your ModelInputBytes class here?

The error basically says that the model can't find a column named "ImageSource", so I would check if ModelInputBytes has that column.

Besides, can you also share which version model builder you are using, the model generated from latest version should use raw byte image input by default and it also has an example of how to create a raw byte image input in the try-out (final) page of wizard

[danieldugger]

Thanks. I based the class off what was listed above. I have also tried ImageSource. Don't know where to find the version of ML.net. The example did show using a physical file vs. memorystream.

//Load sample data var sampleData = new Monkeypox.ModelInput() { ImageSource = @"C:\Users\ddugger\Downloads\MPX Pictures\ML.net\Negative\NM01_01.jpg", };

//Load model and predict output var result = Monkeypox.Predict(sampleData);

[LittleLittleCloud]

You can go to "extensions -> Manage extensions" to find which version of model builder you are using

The code example you share here looks like from a previous version of model builder, and the new version is

//Load sample data
var imageBytes = File.ReadAllBytes(@"C:\Users\miaomiaomiao\Desktop\WeatherData2\Cloudy\cloudy1.jpg");
AzureImage.ModelInput sampleData = new AzureImage.ModelInput()
{
    ImageSource = imageBytes,
};

//Load model and predict output
var result = AzureImage.Predict(sampleData);

which loads images from memory

[danieldugger]

I appreciate it. It looks like I am using the same version as above. Any ideas?

[LittleLittleCloud]

That's not latest version, the latest one is 16.13.9.2231505

You can either update it in extension manager or download from here https://marketplace.visualstudio.com/items?itemName=MLNET.ModelBuilder2022

Notice that you might need to kick off a new training once you update model builder, since the model need to be re-generated for loading image from memory

[danieldugger]

I really appreciate your help!

Daniel Brittain Dugger Founder Bail Bond Software Solutions 615.405.4589 mobile 866.201.5079 toll-free

@.***

From: Xiaoyun Zhang @.> Sent: Wednesday, August 10, 2022 2:02 PM To: dotnet/machinelearning-modelbuilder @.> Cc: Daniel Dugger @.>; Mention @.> Subject: Re: [dotnet/machinelearning-modelbuilder] Predict In-Memory ImageData in Image Classification (#851)

That's not latest version, the latest one is 16.13.9.2231505

You can either update it in extension manager or download from here https://marketplace.visualstudio.com/items?itemName=MLNET.ModelBuilder2022

Notice that you might need to kick off a new training once you update model builder, since the model need to be re-generated for loading image from memory

— Reply to this email directly, view it on GitHubhttps://github.com/dotnet/machinelearning-modelbuilder/issues/851#issuecomment-1211138703, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABMXGWHZRYZQJKE6QYYW5DLVYP4DBANCNFSM4OFMRCDQ. You are receiving this because you were mentioned.Message ID: @.**@.>>

[LittleLittleCloud]

You are welcome, let us know if updating model builder resolve your issue

[danieldugger]

Will do.

Daniel Brittain Dugger Founder Bail Bond Software Solutions 615.405.4589 mobile 866.201.5079 toll-free

@.***

From: Xiaoyun Zhang @.> Sent: Wednesday, August 10, 2022 2:37 PM To: dotnet/machinelearning-modelbuilder @.> Cc: Daniel Dugger @.>; Mention @.> Subject: Re: [dotnet/machinelearning-modelbuilder] Predict In-Memory ImageData in Image Classification (#851)

You are welcome, let us know if updating model builder resolve your issue

— Reply to this email directly, view it on GitHubhttps://github.com/dotnet/machinelearning-modelbuilder/issues/851#issuecomment-1211182283, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABMXGWDY3WMPCXX6QZZN3JTVYQAG7ANCNFSM4OFMRCDQ. You are receiving this because you were mentioned.Message ID: @.**@.>>

[danieldugger]

Thanks, LittleLittleCloud. Being behind in ML.net builder version was the issue.

[rzvme]

I actually have the same issue

ML.NET Model Builder 2022 extension version 16.13.10.2241902

Code:

Models: public class ModelInput { [ColumnName(@"Label")] public string Label { get; set; }

        [ColumnName(@"ImageSource")]
        public string ImageSource { get; set; }

    }
  public class ModelInputBytes
    {
        [ColumnName(@"Label")]
        public string Label { get; set; }

        [ColumnName(@"Features")]
        public byte[] ImageBytes { get; set; }

    }

    public static ITransformer RetrainPipeline(MLContext context, IDataView trainData)
    {

        var dataPreProcess = context.Transforms.Conversion.MapValueToKey(@"Label", @"Label")
                                .Append(context.Transforms.LoadRawImageBytes(@"ImageSource_featurized", @"ImageSource"))
                                .Append(context.Transforms.CopyColumns(@"Features", @"ImageSource_featurized"));
        //var trainingPipeline = context.MulticlassClassification.Trainers.ImageClassification(labelColumnName: @"Label")
        //                        .Append(context.Transforms.Conversion.MapKeyToValue(@"PredictedLabel", @"PredictedLabel"));
        //this is commented because cannot find "ImageClassification"

        var processedData = dataPreProcess.Fit(trainData).Transform(trainData);
        var pipeline = BuildPipeline(context);
        var model = pipeline.Fit(processedData);

        return model;
    }
    public static ModelOutput Predict(ModelInput input)
    {
        MLContext mlContext = new MLContext();

        // Load model & create prediction engine
        ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);

        ITransformer dataPreProcessTransform = LoadImageFromFileTransformer(input, mlContext);

        var predEngine = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(dataPreProcessTransform.Append(mlModel));

        ModelOutput result = predEngine.Predict(input);
        return result;
    }

    public static ITransformer LoadImageFromFileTransformer(ModelInput input, MLContext mlContext)
    {
        var dataPreProcess = mlContext.Transforms.Conversion.MapValueToKey(@"Label", @"Label")
                                .Append(mlContext.Transforms.LoadRawImageBytes(@"ImageSource_featurized", @"ImageSource"))
                                .Append(mlContext.Transforms.CopyColumns(@"Features", @"ImageSource_featurized"));

        var dataView = mlContext.Data.LoadFromEnumerable(new[] { input });
        var dataPreProcessTransform = dataPreProcess.Fit(dataView);
        return dataPreProcessTransform;
    }

    public static ModelOutput PredictFromBytes(ModelInputBytes input)
    {
        MLContext mlContext = new MLContext();

        // Load model & create prediction engine
        ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);

        var predEngine = mlContext.Model.CreatePredictionEngine<ModelInputBytes, ModelOutput>(mlModel);

        ModelOutput result = predEngine.Predict(input);
        return result;
    }

    ERROR: System.ArgumentOutOfRangeException: 'Could not find input column 'ImageSource' (Parameter 'inputSchema')'

[danieldugger]

So, I was able to use the following class to submit to the prediction engine:

public class ModelInput { [ColumnName(@"Label")] public string Label { get; set; } [ColumnName(@"ImageSource")] public byte[] ImageSource { get; set; } }

If you used the Model Builder, you can view the Consume tab and it will give a code snippet showing how to properly construct your code.

[rzvme]

Thx for the reply @danieldugger

I believe my code was copied per the reply from Jake. I did use the model builder, but like a few on this thread, I want to feed images in the form of bytes[] to the predict method, so I did all the changes mentioned above. What I am really interested in is a working solution which would take bytes[], because none of the changes recommended above on the generated code seem to work.

[danieldugger]

Does your Model Builder look like this?

[rzvme]

No, I've already added the console app project to a solution. And also, it seems the builder won't show for since updating to the latest version of the VS extension

[danieldugger]

Ok. Hopefully, someone else can weigh in on the issue.

Daniel Brittain Dugger Founder Bail Bond Software Solutions 615.405.4589 mobile 866.201.5079 toll-free

---

From: rzvme @.> Sent: Sunday, September 11, 2022 2:52:14 AM To: dotnet/machinelearning-modelbuilder @.> Cc: Daniel Dugger @.>; Mention @.> Subject: Re: [dotnet/machinelearning-modelbuilder] Predict In-Memory ImageData in Image Classification (#851)

No, I've already added the console app project to a solution. And also, it seems the builder won't show for since updating to the latest version of the VS extension

— Reply to this email directly, view it on GitHubhttps://github.com/dotnet/machinelearning-modelbuilder/issues/851#issuecomment-1242910243, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABMXGWCMBYIU7HYX2FXTKJ3V5WFS5ANCNFSM4OFMRCDQ. You are receiving this because you were mentioned.Message ID: @.***>

[kimhegg]

No, I've already added the console app project to a solution. And also, it seems the builder won't show for since updating to the latest version of the VS extension

I also strugle with this issue.

@rzvme Did you figure it out?

@danieldugger Should i just be able to feed in a byte[] with ModelInput like this - And no other changes should be required?

 public class ModelInput
    {
        [ColumnName("Label")]
        public string Label { get; set; }

        [ColumnName("ImageSource"))]
        public byte[] ImageSource { get; set; }
    }

[rzvme]

I remember I had a few successful runs a while back but new models (created in the last week) after I've changed the Input class to take byte[] Image Source I am getting this error:

Schema mismatch for input column 'ImageSource': expected String, got Vector (Parameter 'inputSchema')

on: return mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(mlModel);

[LittleLittleCloud]

@rzvme which version of model builder are you using

[rzvme]

Nice catch. I was using 16.13 a while back but recently I swapped laptops and 16.7 comes with the fresh install

[rzvme]

I reinstalled 16.13.10.2241902 and redid the models, I am still getting the same error:

System.ArgumentOutOfRangeException: 'Schema mismatch for input column 'ImageSource': expected image, got Vector (Parameter 'inputSchema')'

on

        return mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(mlModel);

[LittleLittleCloud]

Hmmm I have some trouble reprodcuing this bug. Everything just works fine. Here's my ModelInput class

        public class ModelInput
        {
            [ColumnName(@"Label")]
            public string Label { get; set; }

            [ColumnName(@"ImageSource")]
            public byte[] ImageSource { get; set; }

        }

and the caller for prediction

// Create single instance of sample data from first line of dataset for model input
var imageBytes = File.ReadAllBytes(@"C:\Users\xiaoyuz\Desktop\WeatherData2\Cloudy\cloudy1.jpg");
Local_image.ModelInput sampleData = new Local_image.ModelInput()
{
    ImageSource = imageBytes,
};

// Make a single prediction on the sample data and print results.
var predictionResult = Local_image.Predict(sampleData);