roflcoopter / viseron

Self-hosted, local only NVR and AI Computer Vision software. With features such as object detection, motion detection, face recognition and more, it gives you the power to keep an eye on your home, office or any other place you want to monitor.
MIT License
1.5k stars 154 forks source link

Failed to retrieve result for object_detector #766

Open ogmkp opened 3 weeks ago

ogmkp commented 3 weeks ago

Hello, I have this problem on a NUC, it has Intel GNA.

However, the GPU is fully used on ffmpeg start but takes low usage, no usage for python3 at all but cpu is full throttle on 1 core only, so if you have a solution, It will be great !

So, if you need more infos,

[2024-06-09 03:52:40] [INFO    ] [viseron.components.nvr.nvr.camera_1] - NVR for camera Entrée initialized
[2024-06-09 03:52:40] [INFO    ] [viseron.components] - Setup of domain nvr for component nvr with identifier camera_1 took 0.0 seconds
[2024-06-09 03:52:40] [WARNING ] [viseron.components.ffmpeg.stream.camera_1] - Container mp4 does not support pcm_alaw audio codec. Audio will be transcoded as aac.
[2024-06-09 03:52:40] [INFO    ] [viseron.core] - Viseron initialized in 2.8 seconds
[2024-06-09 03:52:42] [ERROR   ] [viseron.components.ffmpeg.stream.camera_1] - [hevc @ 0x5579c2b0b600] Could not find ref with POC 21
[2024-06-09 03:52:56] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 2 times
[2024-06-09 03:52:59] [ERROR   ] [root] - Uncaught thread exception
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/src/viseron/domains/object_detector/__init__.py", line 402, in _object_detection
    objects = self.return_objects(preprocessed_frame)
  File "/src/viseron/components/darknet/object_detector.py", line 45, in return_objects
    detections = self._darknet.detect(
  File "/src/viseron/components/darknet/__init__.py", line 273, in detect
    return self._model.detect(
cv2.error: OpenCV(4.6.0) /tmp/opencv/modules/dnn/src/ie_ngraph.cpp:747: error: (-2:Unspecified error) Failed to initialize Inference Engine backend (device = GPU): cldnn program build failed! out of GPU resources in function 'initPlugin'

[2024-06-09 03:53:05] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 3 times
[2024-06-09 03:53:07] [ERROR   ] [viseron.watchdog.thread_watchdog] - Thread camera_1.object_detection is dead, restarting
[2024-06-09 03:53:44] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 13 times
[2024-06-09 03:53:46] [ERROR   ] [viseron.components.ffmpeg.stream.camera_1] - [hevc @ 0x5579c2b0b600] Could not find ref with POC 20
[2024-06-09 03:54:51] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 21 times
[2024-06-09 03:54:53] [ERROR   ] [root] - Uncaught thread exception
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/src/viseron/domains/object_detector/__init__.py", line 402, in _object_detection
    objects = self.return_objects(preprocessed_frame)
  File "/src/viseron/components/darknet/object_detector.py", line 45, in return_objects
    detections = self._darknet.detect(
  File "/src/viseron/components/darknet/__init__.py", line 273, in detect
    return self._model.detect(
cv2.error: OpenCV(4.6.0) /tmp/opencv/modules/dnn/src/ie_ngraph.cpp:747: error: (-2:Unspecified error) Failed to initialize Inference Engine backend (device = GPU): cldnn program build failed! out of GPU resources in function 'initPlugin'

[2024-06-09 03:55:06] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 5 times
[2024-06-09 03:55:07] [ERROR   ] [viseron.watchdog.thread_watchdog] - Thread camera_1.object_detection is dead, restarting
[2024-06-09 03:55:22] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 5 times
[2024-06-09 03:55:23] [ERROR   ] [viseron.components.ffmpeg.stream.camera_1] - [hevc @ 0x5579c2b0b600] Could not find ref with POC 0
[2024-06-09 03:56:46] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 25 times
[2024-06-09 03:56:49] [ERROR   ] [root] - Uncaught thread exception
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/src/viseron/domains/object_detector/__init__.py", line 402, in _object_detection
    objects = self.return_objects(preprocessed_frame)
  File "/src/viseron/components/darknet/object_detector.py", line 45, in return_objects
    detections = self._darknet.detect(
  File "/src/viseron/components/darknet/__init__.py", line 273, in detect
    return self._model.detect(
cv2.error: OpenCV(4.6.0) /tmp/opencv/modules/dnn/src/ie_ngraph.cpp:747: error: (-2:Unspecified error) Failed to initialize Inference Engine backend (device = GPU): cldnn program build failed! out of GPU resources in function 'initPlugin'

[2024-06-09 03:56:52] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 2 times
[2024-06-09 03:56:52] [ERROR   ] [viseron.watchdog.thread_watchdog] - Thread camera_1.object_detection is dead, restarting
[2024-06-09 03:57:10] [ERROR   ] [viseron.components.nvr.nvr.camera_1] - Failed to retrieve result for object_detector, message repeated 6 times

556556

6655665

ogmkp commented 3 weeks ago

It works with a better Intel cpu, so I think a small J4005 celeron can't work well.

roflcoopter commented 3 weeks ago

Seems the iGPU runs out of memory. If you want to use the older CPU you can disable running object detection on it by setting dnn_backend: opencv and dnn_target: cpu


darknet:
  object_detector: 
    ... rest of your config
    dnn_backend: opencv
    dnn_target: cpu
ogmkp commented 1 week ago

Thanks for your reply and sorry for my late answer.

It's better, the cpu stills at 50% now on throttle. But the CPU isn't powerful enough anyway, perhaps with further improvements... I'm running Viseron on a more powerful machine in the meantime.