Slow “public void forward(List<Mat> outputBlobs, List<string> outBlobNames)”

[Skodeva]

Summary of your issue

This method will occupy the time of the main thread, resulting in a decrease in the number of frames： public void forward(List<Mat> outputBlobs, List<string> outBlobNames)

Environment

win yolov5 onnx

What did you do when you faced the problem?

When I tested object recognition, I found that the screen was stuck. Using analysis tools, I found that this method would take up the time of the main thread, resulting in a decrease in frame rate. I want to try to put it in a coroutine, but I didn't find a coroutine method that has this method in opencvforunity. I hope to seek help to optimize Caton，thanks~

Example code:

public void forward(List<Mat> outputBlobs, List<string> outBlobNames)
        {
            ThrowIfDisposed();
            Mat outputBlobs_mat = new Mat();
            Mat outBlobNames_mat = Converters.vector_String_to_Mat(outBlobNames);
            dnn_Net_forward_14(nativeObj, outputBlobs_mat.nativeObj, outBlobNames_mat.nativeObj);
            Converters.Mat_to_vector_Mat(outputBlobs_mat, outputBlobs);
            outputBlobs_mat.release();
        }

[AvenSun]

it's hard to say without more details. DNN inference is a computationally intensive operation. generally, there are several ways that can be used to decrease the time of OpenCV's DNN module.

• Use a more efficient hardware, such as high-end GPU.

• Use a smaller and more efficient model. This can be achieved by using techniques such as pruning, quantization, and compression.

• Use batch processing: Performing forward passes on multiple inputs at once can significantly improve the performance of the DNN module.

• Use asynchronous processing.

[Skodeva]

it's hard to say without more details. DNN inference is a computationally intensive operation. generally, there are several ways that can be used to decrease the time of OpenCV's DNN module.

• Use a more efficient hardware, such as high-end GPU.
• Use a smaller and more efficient model. This can be achieved by using techniques such as pruning, quantization, and compression.
• Use batch processing: Performing forward passes on multiple inputs at once can significantly improve the performance of the DNN module.
• Use asynchronous processing.

Can you tell me how to use asynchronous processing? Because the method is not an IEnumerator method, so I don't know how to do it, thanks!

[AvenSun]

one of them is using queue. you can refer to dnn example of opencv

[Skodeva]

one of them is using queue. you can refer to dnn example of opencv

Thank you very much, I can see that this script has some optimization strategies, but the language I use here is C#, so I can't fully understand this script. Excuse me again, can you provide a C# script for my reference, thank you very much!

[Skodeva]

one of them is using queue. you can refer to dnn example of opencv

Hello, I solved this problem with multithreading, thanks! If someone encounters this problem later, it can also be solved with multithreading. The approximate implementation method is to cut the method in Update into the coroutine:

Mat inferResults = null;

IEnumerator Detec(){
...
inferResults = null;
DoInfer(bgrMat);
while (inferResults == null)
      yield return null;
...
}

async void DoInfer(Mat bgrMat)
{
  await Task.Run(() =>
  {
    Mat results = objectDetector.infer(bgrMat);
    inferResults = results;
  });
}

...