How to add new objects for detection?

[zydjohnHotmail]

Hello: I have tested your repo., it works well for already defined objects, like people. But in my image, I want to detect a logo, you can see the part I circled with a red box. By the way, I also want to detect other objects, one is score board, if you see the basketball game, you know what scoreboard is. Another is a clock, like in the train station or other public transport station, people will see the time on a clock to know when the train will leave or arrive. Please advise on how to add some train dataset for those objects I am interested, like the logo in the picture and a clock. See the picture:

Thanks, PS: I am using Visual Studio 2019 (Version 16.11.2) on Windows 10. One minor issue: the goal keeper who fall on the ground was not detected, how to improve the code?

[kkarahainko]

@zydjohnHotmail hi, yolov5-net is only an "inference wrapper" over ultrlytics/yolov5, so to train a custom model please look at https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data it's quite a simple task, you may use Google Colab, Roboflow to speed up training, and LabelBox for annotations.

[zydjohnHotmail]

Hello: Thanks for your reply. I will spend some time to understand how to train a custom model. However, what I want to know is: since all the training is in Python, after I trained my custom model, how I can use the result back in yolov5-net? I see your code, under Assets, there is one weight folder, and there are 2 files: yolov5s.onnx and yolov5s6.onnx What I should do to convert what kind of result after training to be used by yolov5-net? Please provide some instructions. Thanks,

[kkarahainko]

@zydjohnHotmail after training just export model from .pt to .onnx https://github.com/ultralytics/yolov5/issues/251, single code line python export.py --weights custom_model.pt --img 640 --batch 1, then use it from C# (with yolov5-net).

[zydjohnHotmail]

Hello: I have some hard time to train a small dataset with only 3 images and one label. From your experience, do I have to install a whole bunch stuff like: CUDA and CUDNN? I have a powerful CPU: AMD Threadripper with 16-core CPU and 128GB RAM, for most of the heavy-computing job, using CPU is enough. And the size of CUDA is amazing: 2.8GB. Can I train such small dataset without installing all those heavy weapons? When I run the following command to train my small dataset: C:\Videos\ultralytics\yolov5>python train.py --img 3 --batch 3 --epochs 3 --data logo.yaml --weights '' --cfg yolov5s.pt I got error: AssertionError: yolov5s.pt acceptable suffix is ('.yaml', '.yml') I don't understand it. Please advise! Thanks,

[kkarahainko]

@zydjohnHotmail honestly, you don't need anything locally on your PC, just use Google Colab, it provides GPU env with filesystem for free. To train just use python train.py --img 640 --batch 16 --epochs 3 --data logo.yaml --weights yolov5s.pt without --cfg etc. Use this as example https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb.

[kkarahainko]

@zydjohnHotmail, of course, you need more epochs, about 500 - 1000 (not 3) 😄

[zydjohnHotmail]

Hello: Thank you very much for your advice, the following is my logo.yaml file:

C:\Videos\ultralytics\yolov5>type logo.yaml

Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]

path: ../datasets/logo # dataset root dir train: ../datasets/logo/images/logo3 # train images (relative to 'path') val: ../datasets/logo/images/logo3 # train images (relative to 'path')

Classes

nc: 1 names: [ 'logo' ]

I tried the following train command. (The image size is: 720 by 406 pixel) D:\Videos\ultralytics\yolov5>python train.py --img 720 --batch 16 --epochs 30 --data ./logo.yaml --cfg yolov5s.yaml --weights yolov5s.pt ...... But I got error: AssertionError: train: No labels in ..\datasets\logo..\datasets\logo\labels\logo3.cache. Can not train without labels. See https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data

I can't figure out what is wrong with my logo.yaml file. Please advise! Thanks,

[zydjohnHotmail]

Hello: Finally, I got a small dataset trained and get the export model from .pt to .onnx But how I can change the code in C#: using var image = Image.FromFile(Image_With_Logo); using var scorer = new YoloScorer("Assets/Weights/best.onnx"); List predictions = scorer.Predict(image); using var graphics = Graphics.FromImage(image);

The best.onnx is the exported trained model. But when I run my code, I got error: Microsoft.ML.OnnxRuntime.OnnxRuntimeException HResult=0x80131500 Message=[ErrorCode:InvalidArgument] Got invalid dimensions for input: images for the following indices index: 2 Got: 640 Expected: 768 index: 3 Got: 640 Expected: 768 Please fix either the inputs or the model. Source=Microsoft.ML.OnnxRuntime StackTrace: at Microsoft.ML.OnnxRuntime.NativeApiStatus.VerifySuccess(IntPtr nativeStatus) at Microsoft.ML.OnnxRuntime.InferenceSession.RunImpl(RunOptions options, IntPtr[] inputNames, IntPtr[] inputValues, IntPtr[] outputNames, DisposableList1 cleanupList) at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs, IReadOnlyCollection1 outputNames, RunOptions options) at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs, IReadOnlyCollection1 outputNames) at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs) at Yolov5Net.Scorer.YoloScorer1.Inference(Image image) at Yolov5Net.Scorer.YoloScorer1.Predict(Image image) at Yolov5NetDetectOBJForm.Form1.ButtonDetect_Click(Object sender, EventArgs e) in C:\SoccerVideos\GUIHelper\Yolov5NetDetectOBJForm\Yolov5NetDetectOBJForm\Form1.cs:line 31 at System.Windows.Forms.Control.OnClick(EventArgs e) at System.Windows.Forms.Button.OnClick(EventArgs e) at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent) at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.ButtonBase.WndProc(Message& m) at System.Windows.Forms.Button.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, WM msg, IntPtr wparam, IntPtr lparam)

The image size for train dataset is 768px by 432px. But it seems the YoloCocoP5Model is 640px by 640px. How I can create a new YoloXXX_Model, so it size is 768px? Or I can change training command to use --img 640? Please advise! thanks,

[kkarahainko]

@zydjohnHotmail inherit your custom model class from YoloModel and override required properties (width, height etc) like this https://github.com/mentalstack/yolov5-net/blob/master/src/Yolov5Net.Scorer/Models/YoloCocoP5Model.cs.

[zydjohnHotmail]

Hello: Thank you very much for your reply. I am trying to figure out how to do what you have said. I have done the following: I created one WinForms App project with Visual Studio 2019 named “Yolov5NetDetectOBJForm” (target .NET 5.0). Add one PictureBox to show the PNG image with 768px by 432px, and one button: ButtonDetect, when I click on it, it will run object detect by the old yolov5.onnx. I added one class YoloCocoP5Model.cs with the following code: `using System.Collections.Generic; using Yolov5Net.Scorer; using Yolov5Net.Scorer.Models.Abstract;

namespace Yolov5NetDetectOBJForm { public class YoloCocoP5Model : YoloModel { public override int Width { get; set; } = 768; public override int Height { get; set; } = 432; public override int Depth { get; set; } = 3;

    public override int Dimensions { get; set; } = 85;

    public override float[] Strides { get; set; } = new float[] { 8, 16, 32 };

    public override float[][][] Anchors { get; set; } = new float[][][]
    {
        new float[][] { new float[] { 010, 13 }, new float[] { 016, 030 }, new float[] { 033, 023 } },
        new float[][] { new float[] { 030, 61 }, new float[] { 062, 045 }, new float[] { 059, 119 } },
        new float[][] { new float[] { 116, 90 }, new float[] { 156, 198 }, new float[] { 373, 326 } }
    };

    public override int[] Shapes { get; set; } = new int[] { 80, 40, 20 };

    public override float Confidence { get; set; } = 0.20f;
    public override float MulConfidence { get; set; } = 0.25f;
    public override float Overlap { get; set; } = 0.45f;

    public override string[] Outputs { get; set; } = new[] { "output" };

    public override List<YoloLabel> Labels { get; set; } = new List<YoloLabel>()
    {
        new YoloLabel { Id = 1, Name = "person" },
    };

    public override bool UseDetect { get; set; } = true;

    public YoloCocoP5Model()
    {

    }
}

} `

I want to simply change the Width and Height from 640 to 768 and 432 respectively. Then in Form1.cs, I have this code:

`using System; using System.Collections.Generic; using System.Drawing; using System.Windows.Forms; using Yolov5Net.Scorer; using static Yolov5NetDetectOBJForm.YoloCocoP5Model;

namespace Yolov5NetDetectOBJForm { public partial class Form1 : Form { public const string Image_With_Logo = @"C:\Videos\Logo.PNG";

    public Form1()
    {
        InitializeComponent();

    }

    private void Form1_Load(object sender, EventArgs e)
    {
        Image source_bmp = Image.FromFile(Image_With_Logo);
        pictureBox1.Image = source_bmp;
    }

    private void ButtonDetect_Click(object sender, EventArgs e)
    {
        using var image = Image.FromFile(Image_With_Logo);
        using var scorer = new YoloScorer<YoloCocoP5Model>("Assets/Weights/yolov5s.onnx");
        //using var scorer = new YoloScorer<YoloCocoP5Model>("Assets/Weights/best.onnx");
        List<YoloPrediction> predictions = scorer.Predict(image);
        using var graphics = Graphics.FromImage(image);
        foreach (var prediction in predictions) // iterate predictions to draw results
        {
            double score = Math.Round(prediction.Score, 2);
            graphics.DrawRectangles(new Pen(prediction.Label.Color, 1),
                new[] { prediction.Rectangle });
            var (x, y) = (prediction.Rectangle.X - 3, prediction.Rectangle.Y - 23);
            graphics.DrawString($"{prediction.Label.Name} ({score})",
                new Font("Consolas", 16, GraphicsUnit.Pixel), new SolidBrush(prediction.Label.Color),
                new PointF(x, y));
        }
        image.Save("Assets/result.jpg");
        Console.Beep();
    }
}

} `

Here, I only change the width and height for model, and use only the first label class: person. The image is the one you see from my first post in this issue thread, there are totally 9 people in the image, even the yolov5s.onnx has detected only 5, but it is OK. However, when I run the above code, I got the following errors: Exception thrown: 'Microsoft.ML.OnnxRuntime.OnnxRuntimeException' in Microsoft.ML.OnnxRuntime.dll An exception of type 'Microsoft.ML.OnnxRuntime.OnnxRuntimeException' occurred in Microsoft.ML.OnnxRuntime.dll but was not handled in user code [ErrorCode:InvalidArgument] Got invalid dimensions for input: images for the following indices index: 2 Got: 432 Expected: 640 index: 3 Got: 768 Expected: 640 Please fix either the inputs or the model.

Therefore, I think maybe I changed the wrong class or have other issues. Please advice. You can try to download the picture from my fist post and see if you can use the yolov5s.onnx to detect the people from the image. Thanks,

[kkarahainko]

@zydjohnHotmail can you provide me with your custom trained onnx file?

[kkarahainko]

@zydjohnHotmail if you trained your model on 640 x 640 images then width and height in model class should be 640 x 640, not 768 x 462, all labels should be defined as well, not only person.

[zydjohnHotmail]

Hello: I found the major issue is that my onnx file is not suitable to be used in yolov5-net. To train the model seems not easy. At first, I trained my model using only 30 images with only one label "logo", but when I used my trained model, I can't detect any logo object. Later, I trained my model with more than 2000 images, it took nearly one day, but I didn't have time to test it. I can give it to you for testing, but I don't know how to upload it here.

[kkarahainko]

@zydjohnHotmail sad to hear, maybe something wrong with a training config or dataset, I've tested on different projects and more complex tasks, and it worked as expected.