Performance and yoloWrapper ('Attempted to read or write protected memory)

[pjsgsy]

I was looking ot see if I could improve the performance of some of the common examples. I posted in the original, but I realise this Fork is actually the one I am using from nuget.

One of the notes in the main doc page says " It is also important not to instantiate the wrapper over and over again.". Which is actually one of the first things I have tried. However, if you do not instantiate a new YoloWrapper every time before you do a .Detect with it, you get this error "System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that another memory is corrupt.'"

So, it's required to do yoloWrapper = new YoloWrapper(config); every time, even though the docs are saying this is wasteful (which it is!)

Does anyone know how to make this work whilst re-using the already initialised YolowWrapper?

[zgabi]

Could you please create a small demo app to reproduce the problem? I had no issues with re-using the wrapper.

[pjsgsy]

Thanks for the quick reply.

This is the current full code. I tried auto config and manual config (as it is now). Same result. The detect call 'var items = yoloWrapper.Detect(ms.ToArray()).ToList();' is in the Detect() function. Unless I am doing something stupid, I am not sure why this is not working. I am initialising the wrapper in the Form1_Load event. If I do this also inside the Detect() function, all is fine (but slow) - It's only when trying to re-use it I get the issue.

`using Alturos.Yolo; using System.Diagnostics; using System.Drawing.Drawing2D; using System.Drawing.Imaging; using System.Drawing.Text; using System.Linq.Expressions; using System.Windows.Forms; using AForge.Video; using AForge.Video.DirectShow; using Emgu.CV; using Alturos.Yolo.Model;

namespace ObjectDetection { public partial class Form1 : Form { // EMGU VideoCapture capture; // Aforge FilterInfoCollection filterInfoCollection; VideoCaptureDevice videoCaptureDevice; // YOLO YoloConfigurationDetector configurationDetector = new YoloConfigurationDetector(); YoloConfiguration config; YoloWrapper yoloWrapper; // Other bool NewImage = false; bool Processing = false; bool LiveVideo = false; public Form1() { InitializeComponent(); }

    private void Run()
    {
        if (cmbCameras.SelectedIndex < 0)
        {
            MessageBox.Show("Select a video source from the drop down list first", "No video source", MessageBoxButtons.OK, MessageBoxIcon.Exclamation);
            return;
        }
        //videoCaptureDevice = new VideoCaptureDevice(filterInfoCollection[cmbCameras.SelectedIndex].MonikerString);
        //videoCaptureDevice.NewFrame += VideoCaptureDevice_NewFrame;
        //videoCaptureDevice.Start();
        try
        {
            capture = new VideoCapture(cmbCameras.SelectedIndex);
        }
        catch (Exception Ex)
        {
            MessageBox.Show(Ex.Message);
            return;
        }
        Application.Idle += ProcessFrame;
        LiveVideo = true;
        btnLiveVideo.Text = "&Stop";
    }

    private void ProcessFrame(object sender, EventArgs e)
    {
        var img = capture.QuerySmallFrame();
        if (img != null)
        {
            picImage.Image = img.ToBitmap();
            Detect();
        }
    }
    private void Detect()
    {
        using (yoloWrapper)
        {
            using (MemoryStream ms = new MemoryStream())
            {
                picImage.Image.Save(ms, ImageFormat.Png);
                var items = yoloWrapper.Detect(ms.ToArray()).ToList(); // this will fail after first use with System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'
                AddDetailsToPictureBox(picImage, items);
                //yoloWrapper.Dispose();
            }
        }
    }
    private void AddDetailsToPictureBox(PictureBox picturebox1, List<YoloItem> items)
    {
        var img = picImage.Image;
        var font = new Font("Arial", 12, FontStyle.Bold);
        var brush = new SolidBrush(Color.LimeGreen);
        var graphics = Graphics.FromImage(img);
        foreach (var item in items)
        {

            var x = item.X; var y = item.Y;
            var width = item.Width;
            var height = item.Height;
            var T = item.Type; var C = item.Confidence;
            var rect = new Rectangle(x, y, width, height);
            var pen = new Pen(Color.LightGreen, 3);
            var point = new Point(x, y);
            Debug.WriteLine(C + " : " + T);
            graphics.DrawRectangle(pen, rect);
            graphics.DrawString(T, font, brush, point);
            picImage.Image = img;
        }
    }

    private void Form1_Load(object sender, EventArgs e)
    {
        filterInfoCollection = new FilterInfoCollection(FilterCategory.VideoInputDevice);
        foreach (FilterInfo filterInfo in filterInfoCollection)
            cmbCameras.Items.Add(filterInfo.Name);
        // config = configurationDetector.Detect(@".\assets");
        // yoloWrapper = new YoloWrapper(config);
        yoloWrapper = new YoloWrapper(@".\assets\yolov3.cfg", @".\assets\yolov3.weights", @".\assets\coco.names");
    }

    private void button1_Click(object sender, EventArgs e)
    {
        var ofd = new OpenFileDialog();
        ofd.Filter = "Image Files|*.jpg;*.png";
        if (ofd.ShowDialog() == DialogResult.OK)
        {
            picImage.Image = Image.FromFile(ofd.FileName);
        }

    }

    private void btnDetect_Click(object sender, EventArgs e)
    {
        Detect();
    }
    private void button2_Click(object sender, EventArgs e)
    {
        if (!LiveVideo)
        {
            Run();
        }
        else
        {
            Application.Idle -= ProcessFrame;
            btnLiveVideo.Text = "&Live video";
            LiveVideo = false;
        }
    }
    private void VideoCaptureDevice_NewFrame(object sender, NewFrameEventArgs eventArgs)
    {
        picImage.Image = (Bitmap)eventArgs.Frame.Clone();
        Detect();
    }

    private void Form1_FormClosing(object sender, FormClosingEventArgs e)
    {
        if (videoCaptureDevice != null && videoCaptureDevice.IsRunning)
            videoCaptureDevice.Stop();

    }

}

}`

[pjsgsy]

Here is the entire project (minus the assets folder / cuda dll and yolov3 files from the assets folder to keep the size down) Program.zip

[pjsgsy]

Slightly more info, from the output console

An unhandled exception of type 'System.AccessViolationException' occurred in Alturos.Yolo.dll Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

I am continuing to try and figure it out. I will update if I find a solution, but am not sure it is something I can resolve.

[zgabi]

Which CUDA version do you have?

[zgabi]

Is it throwing the exception when the videoCaptureDevice things are commented out, too? So only when you press the button.

(I'm asking because I suspect that the VideoCaptureDevice_NewFrame is called parallel.. so the previos Decode is still unning when a new frame is available. Maybe you should try to add a lock around the Decode method)

[pjsgsy]

I don't think it is a thread/lock-type issue. The same code allows me to select a static picture and try the detect that. If I select a picture (no video running), click detect, it works. Click detect once more, same error.

I think when I first tried this I was using CPU, not CUDA,, but I am re-installing all the CUDA bits now to try. I tried some locks around the code anyway. Same. Will update if I can figure it out.

[pjsgsy]

I might have resolved it. 2 things. I had a 'using yolowrapper there. Maybe I was disposing it after use inadvertently... But, I will had 'object in use' issues after that. I changed this code

pictureBox1.Invoke(new MethodInvoker( delegate () { pictureBox1.Image.Save(ms, ImageFormat.Png); }));

So, pinvoke around the image.save on the picture box which would have been in the UI thread. I've got it running at 4 frames a second now.

Thanks for your help. This was surprisingly tough to get running even with the few samples I found! I will try and share the completed, working project somewhere!