ONNX Vision

[hannespreishuber]

.NET 8 ML.NET 3

running out of ideas exception

did a training with customvision.ai - exported and download the onnx.model. Use the ZIP attached cs file and created a UWP app within minutes- works.

Try to rebuild this stuff with console app. Nothing detected Repro https://github.com/hannespreishuber/MLNETONNX

using Microsoft.ML.Data;
using Microsoft.ML.Transforms.Image;
using Microsoft.ML.Transforms.Onnx;
using MLNETONNX;

string assetsPath = @"C:\\stateof\\MLNETONNX\\MLNETONNX\\assets";
var modelFilePath = Path.Combine(assetsPath, "model.onnx");

var mlContext = new MLContext();

//var imageData = new ImageData
//{
//    Image = File.ReadAllBytes(@"c:\temp\test.png"),
//    ImagePath = @"c:\temp\test.png"
//};

//B
var imageData = new ImageData
{
    Image = MLImage.CreateFromFile(@"c:\temp\test.png")

};
var dataView = mlContext.Data.LoadFromEnumerable(new List<ImageData>() );

//var pipeline =
//    mlContext.Transforms.LoadImages(outputColumnName: "data", imageFolder: "", inputColumnName: nameof(ImageData.ImagePath))
//    .Append(mlContext.Transforms.ResizeImages(
//                   inputColumnName: "data",
//                   outputColumnName: "data",
//                   imageWidth: 416,
//                   imageHeight: 416,
//                   resizing: ImageResizingEstimator.ResizingKind.Fill))
//    .Append(mlContext.Transforms.ExtractPixels(inputColumnName:"data",
//                   outputColumnName: "data"))
//    .Append(mlContext.Transforms.ApplyOnnxModel(modelFile: modelFilePath,
//    outputColumnNames: new[] { "model_outputs0" }, 
//      inputColumnNames: new[] { "data" }));

//B
var pipeline =  mlContext.Transforms.ResizeImages("data", 416,416, nameof(ImageData.Image))
        .Append(mlContext.Transforms.ExtractPixels( "data",  "data"))
        .Append(mlContext.Transforms.ApplyOnnxModel("model_outputs0", "data", modelFilePath));

var model = pipeline.Fit(dataView);

var predictionEngine = mlContext.Model.CreatePredictionEngine<ImageData, List<PredictionModel>>(model);
var prediction = predictionEngine.Predict(imageData);

Console.WriteLine($"Vorhersage: ");

public sealed class BoundingBox
{
    public BoundingBox(float left, float top, float width, float height)
    {
        this.Left = left;
        this.Top = top;
        this.Width = width;
        this.Height = height;
    }

    public float Left { get; private set; }
    public float Top { get; private set; }
    public float Width { get; private set; }
    public float Height { get; private set; }
}

public sealed class PredictionModel
{
    public PredictionModel(float[] probability, string tagName, BoundingBox boundingBox)
    {
        this.Probability = probability;
        this.TagName = tagName;
        this.BoundingBox = boundingBox;
    }

    public float[] Probability { get; private set; }
    public string TagName { get; private set; }
    public BoundingBox BoundingBox { get; private set; }
}

//public class ImageData
//{
//    [ImageType(416, 416)]
//    [ColumnName("data")]
//    public byte[] Image { get; set; }
//    public string ImagePath { get; set; }

//}

//B
public class ImageData
{
    [ImageType(416, 416)]
    public MLImage Image { get; set; }

}`

[luisquintanilla]

Not sure what the issue may be. One thing I noticed is the size of your image. I think for image classification the size is 300x300 and object detection 320x320. You can check out these samples for additional guidance.

https://github.com/Azure-Samples/customvision-export-samples/tree/main/samples/csharp/mlnet

[luisquintanilla]

Closing issue. Let us know if you run into additional issues or have questions.