search

dme-compunet / YoloV8

Integrate YOLOv8 into your C# project for a variety of real-time tasks including object detection, instance segmentation, pose estimation and more, using ONNX Runtime.
https://www.nuget.org/packages/YoloV8
GNU Affero General Public License v3.0
242 stars 45 forks source link

Solving problems with multithreading when working with GPU #26

Closed oneprofittips closed 9 months ago

oneprofittips commented 10 months ago

Good day, to address the issue of model failure in multi-threaded code without losing performance, I suggest wrapping the code segment where using var outputs = _inference.Run(inputs); is called in a SemaphoreSlim. In the constructor, accept an optional parameter that initializes SemaphoreSlim when working with GPU. No errors have been observed when working with CPU, and the model does not crash. This method would help increase the performance of the library. At the moment, on GPU in single-threaded mode, it processes 30 images per second. I believe that performance can be increased to even higher values.

dme-compunet commented 10 months ago

Thanks for the offer, I'll deal with it as soon, would you be willing to submit a PR with the changes in the meantime?

dme-compunet commented 9 months ago

What are you trying to achieve with SemaphoreSlim, why not use lock block?

oneprofittips commented 9 months ago

I'm trying to get the most out of the GPU. The current configuration has multithreaded code. causes a qnx runtime error, it is necessary to start 1 thread in the code sector that will work with the GPU, and let it prepare images and post-processing in a multi-thread.

dme-compunet commented 9 months ago

I pushed an update recently, check if this commit does what you need. https://github.com/dme-compunet/YOLOv8/commit/555dac6a95aa31c38f55e9e7e4552b7086b60e18

oneprofittips commented 9 months ago

Yes, thank you. I got about a 30% increase. Is it possible to send packets for detection? (batchs)

dme-compunet commented 9 months ago

at the moment, no..