search

dme-compunet / YoloV8

Integrate YOLOv8 into your C# project for a variety of real-time tasks including object detection, instance segmentation, pose estimation and more, using ONNX Runtime.
https://www.nuget.org/packages/YoloV8
GNU Affero General Public License v3.0
242 stars 45 forks source link

Multithreaded execution error. #21

Closed oneprofittips closed 11 months ago

oneprofittips commented 11 months ago

Good afternoon. Multithreaded handling of DetectAsync and Detect in GPU mode causes an error and detection no longer works.

2023-09-24 20:39:41.0146901 [E:onnxruntime:CSharpOnnxRuntime, cuda_call.cc:116 onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\gpu_data_transfer.cc ; line=63 ; expr=cudaMemcpyAsync(dst_data, src_data, bytes, cudaMemcpyHostToDevice, static_cast<cudaStream_t>(stream.GetHandle())); 2023-09-24 20:39:41.0256717 [E:onnxruntime:CSharpOnnxRuntime, cuda_call.cc:116 onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=409 ; expr=cudaStreamSynchronize(static_cast<cudaStream_t>(stream_)); 2023-09-24 20:39:41.0299946 [E:onnxruntime:CSharpOnnxRuntime, cuda_call.cc:116 onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\gpu_data_tError: Microsoft.ML.OnnxRuntime.OnnxRuntimeException: [ErrorCode:Fail] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a\_work\1\s\onnxruntime\core\providers\cuda\gpu_data_transfer.cc ; line=63 ; exprransfer.cc ; line=63 ; expr=c=cudaMemcpyAsync(dst_data, src_data, bytes, cudaMemcpyHostToDevice, static_cast<cudaStream_t>(stream.GetHandle())); at Microsoft.ML.OnnxRuntime.NativeApiStatus.VerifySuccess(IntPtr nativeStatus) at Microsoft.ML.OnnxRuntime.InferenceSession.RunImpluda(RunOptions options, IntPtr[] inputNames, IntPtr[] inputValues, IntPtr[] outputNames, DisposableList1 cleanupList) at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs, IReadOnlyCollection1 outputNames, RunOptions options) at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs, IReadOnlyCollection1 outputNames) at Microsoft.ML.OnnxRuntime.InferenceSession.Run(IReadOnlyCollection1 inputs) at Compunet.YoloV8.YoloV8.Run[TResult](ImageSelector selector, PostprocessContext1 postprocess) at Compunet.YoloV8.YoloV8Extensions.Detect(YoloV8 predictor, ImageSelector selector) at Compunet.YoloV8.YoloV8Extensions.<>cDisplayClass5_0.b__0() at System.Threading.Tasks.Task`1.InnerMemcpyAsync(dst_data, src_data, bytes, Invoke() at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- at System.Threading.EcudaMemcpyHostToDevice, static_cast<cudaStreaxecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) --- End of stack trace from previous location --- at Compunet.YoloV8.YoloV8Extensions.DetectAsync(YoloV8 predictor, ImageSelector selector) at StressTest.DetectService.DetectAsync(String base64Img) in D:\Shared\sharpprogects\VisionSystems\StressTest\DetectService.cs:line 50 at StressTest.Program.<>cDisplayClass1_0.<b__3>d.MoveNext() in D:\Shared\sharpprogects\VisionSystems\StressTest\Program.cs:line 91 m_t>(stream.GetHandle())); 2023-09-24 20:39:41.0649941 [E:onnxruntime:CSharpOnnxRuntime, cuda_call.cc:116 onnxruntime::CudaCall] CUDA failure 700: an illegal memory access was encountered ; GPU=0 ; hostname=DESKTOP-39DKFBQ ; file=D:\a_work\1\s\onnxruntime\core\providers\cuda\cuda_execution_provider.cc ; line=409 ; expr=cudaStreamSynchronize(static_cast(stream_));`

dme-compunet commented 11 months ago

It seems that ONNX-Runtime is unable to handle Multithreading in GPU mode, so it's not a problem with this repo.

Try opening an issue about it here: https://github.com/microsoft/onnxruntime