microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.15k stars 2.86k forks source link

[C#] Invalid input name error #17244

Open Day-commit opened 1 year ago

Day-commit commented 1 year ago

Describe the issue

I want to use C# to call yolov8 to convert the onnx model file, and when loading, it prompts Microsoft.ML.OnnxRuntime.OnnxRuntimeException: "[ErrorCode:InvalidArgument] Input name: 'image' is not in the metadata".how to solve this problem.Below is my code:

To reproduce

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Microsoft.ML.OnnxRuntime.Tensors;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats;
using SixLabors.ImageSharp.PixelFormats;
using SixLabors.ImageSharp.Processing;
using SixLabors.ImageSharp.Drawing.Processing;
using SixLabors.Fonts;
//using SixLabors.Fonts;
//using System.Drawing;

namespace Microsoft.ML.OnnxRuntime.FasterRcnnSample
{
    class Program
    {
        public static void Main(string[] args)
        {

            // Read paths
            string modelFilePath = @"C:\Users\AMD\Desktop\ultralytics-main\runs\segment\train4\weights\best.onnx";
            //string modelFilePath = @"D:\Test\2022\best.onnx";
            // string imageFilePath = @"D:\Test\2022\A.jpg";
            string imageFilePath = @"C:\Users\AMD\Desktop\ultralytics-main\testimage\Cam0_OK__16_27_12.900_1098.png";
            string outImageFilePath = "outputs.jpg";
            //System.IO.Directory.CreateDirectory(outImageFilePath);

            // Read image
            using Image<Rgb24> image = SixLabors.ImageSharp.Image.Load<Rgb24>(imageFilePath);

            // Resize image
            float ratio = 800f / Math.Min(image.Width, image.Height);
            image.Mutate(x => x.Resize((int)(ratio * image.Width), (int)(ratio * image.Height)));

            // Preprocess image
            var paddedHeight = (int)(Math.Ceiling(image.Height / 32f) * 32f);
            var paddedWidth = (int)(Math.Ceiling(image.Width / 32f) * 32f);
            Tensor<float> input = new DenseTensor<float>(new[] { 3, paddedHeight, paddedWidth });
            var mean = new[] { 102.9801f, 115.9465f, 122.7717f };
            for (int y = paddedHeight - image.Height; y < image.Height; y++)
            {
                image.ProcessPixelRows(im =>
                {
                    var pixelSpan = im.GetRowSpan(y);
                    for (int x = paddedWidth - image.Width; x < image.Width; x++)
                    {
                        input[0, y, x] = pixelSpan[x].B - mean[0];
                        input[1, y, x] = pixelSpan[x].G - mean[1];
                        input[2, y, x] = pixelSpan[x].R - mean[2];
                    }
                });

            }

            // Setup inputs and outputs
            var inputs = new List<NamedOnnxValue>
            {
                NamedOnnxValue.CreateFromTensor("image", input)
            };

            // Run inference
            using var session = new InferenceSession(modelFilePath);
            using IDisposableReadOnlyCollection<DisposableNamedOnnxValue> results = session.Run(inputs);

            // Postprocess to get predictions
            var resultsArray = results.ToArray();
            float[] boxes = resultsArray[0].AsEnumerable<float>().ToArray();
            long[] labels = resultsArray[1].AsEnumerable<long>().ToArray();
            float[] confidences = resultsArray[2].AsEnumerable<float>().ToArray();

            var predictions = new List<Prediction>();
            var minConfidence = 0.7f;
            for (int i = 0; i < boxes.Length - 4; i += 4)
            {
                var index = i / 4;
                if (confidences[index] >= minConfidence)
                {
                    predictions.Add(new Prediction
                    {
                        Box = new Box(boxes[i], boxes[i + 1], boxes[i + 2], boxes[i + 3]),
                        Label = LabelMap.Labels[labels[index]],
                        Confidence = confidences[index]
                    });
                }
            }

            // Put boxes, labels and confidence on image and save for viewing
            using var outputImage = File.OpenWrite(outImageFilePath);
            Font font = SystemFonts.CreateFont("Arial", 16); 
            var outputFilePath = "OutPutImage.jpg";

            using (var image2 = Image.Load(imageFilePath))
            {
               // var predictions = GetPredictions(); //  

                foreach (var p in predictions)
                {
                    image2.Mutate(x =>
                    {
                        var pen = Pens.Solid(Color.Red, 2f);
                        x.Draw(pen, new RectangleF(p.Box.Xmin, p.Box.Ymin, p.Box.Xmax - p.Box.Xmin, p.Box.Ymax - p.Box.Ymin));

                        var font = SystemFonts.CreateFont("Arial", 12f, FontStyle.Regular);
                        var textPosition = new PointF(p.Box.Xmin, p.Box.Ymin);
                        x.DrawText($"{p.Label}, {p.Confidence:0.00}", font, Color.White, textPosition);
                    });
                }

                image2.Save(outputFilePath); //  
            }
            //foreach (var p in predictions)
            //{
            //    image.Mutate(x =>
            //    {
            //        x.DrawLines(Color.Red, 2f, new PointF[] {

            //            new SixLabors.ImageSharp.PointF(p.Box.Xmin, p.Box.Ymin),
            //            new SixLabors.ImageSharp.PointF(p.Box.Xmax, p.Box.Ymin),

            //            new PointF(p.Box.Xmax, p.Box.Ymin),
            //            new PointF(p.Box.Xmax, p.Box.Ymax),

            //            new PointF(p.Box.Xmax, p.Box.Ymax),
            //            new PointF(p.Box.Xmin, p.Box.Ymax),

            //            new PointF(p.Box.Xmin, p.Box.Ymax),
            //            new PointF(p.Box.Xmin, p.Box.Ymin)
            //        });
            //        x.DrawText($"{p.Label}, {p.Confidence:0.00}", font, Color.White, new PointF(p.Box.Xmin, p.Box.Ymin));
            //    });
            //}
            image.SaveAsJpeg(outputImage);
        }
    }
    public class Prediction
    {
        public Box Box { set; get; }
        public string Label { set; get; }
        public float Confidence { set; get; }
    }
    public class Box
    {
        public Box(float xMin, float yMin, float xMax, float yMax)
        {
            Xmin = xMin;
            Ymin = yMin;
            Xmax = xMax;
            Ymax = yMax;
        }
        public float Xmin { set; get; }
        public float Xmax { set; get; }
        public float Ymin { set; get; }
        public float Ymax { set; get; }
    }

    public static class LabelMap
    {
        static LabelMap()
        {
            Labels = new string[]
            {
             "",
            "person",
            "bicycle",
            "car",
            "motorcycle",
            "airplane",
            "bus",
            "train",
            "truck",
            "boat",
            "traffic light",
            "fire hydrant",
            "stop sign",
            "parking meter",
            "bench",
            "bird",
            "cat",
            "dog",
            "horse",
            "sheep",
            "cow",
            "elephant",
            "bear",
            "zebra",
            "giraffe",
            "backpack",
            "umbrella",
            "handbag",
            "tie",
            "suitcase",
            "frisbee",
            "skis",
            "snowboard",
            "sports ball",
            "kite",
            "baseball bat",
            "baseball glove",
            "skateboard",
            "surfboard",
            "tennis racket",
            "bottle",
            "wine glass",
            "cup",
            "fork",
            "knife",
            "spoon",
            "bowl",
            "banana",
            "apple",
            "sandwich",
            "orange",
            "broccoli",
            "carrot",
            "hot dog",
            "pizza",
            "donut",
            "cake",
            "chair",
            "couch",
            "potted plant",
            "bed",
            "dining table",
            "toilet",
            "tv",
            "laptop",
            "mouse",
            "remote",
            "keyboard",
            "cell phone",
            "microwave",
            "oven",
            "toaster",
            "sink",
            "refrigerator",
            "book",
            "clock",
            "vase",
            "scissors",
            "teddy bear",
            "hair drier",
            "toothbrush"
            };
        }

        public static string[] Labels { set; get; }
    }
}

Urgency

No response

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

Microsoft.ML.OnnxRuntime1.15.1,Microsoft.ML.OnnxRuntime.Managed1.15.1

Execution Provider

Other / Unknown

fs-eire commented 1 year ago

I assume that the error is as explained in the error message: "image" may not be the model's input

Day-commit commented 1 year ago

我假设错误如错误消息中所述:“图像”可能不是模型的输入

Then what should I do? This problem has troubled me for a long time. Using netron to view the model, I found that the input of the model exported by yolo is images, and using the FasterRCNN-10.onnx model provided by you, it can run normally. His in netron The input is an image, is there any difference between this?

Ishan-Sahu-01 commented 1 year ago

The it should be "images" in NamedOnnxValue.CreateFromTensor("image", input). Or you can check the exact name by using Console.WriteLine(session.InputNames[0]), which will give you the actual name for it and then you can replace it with your "image"