search

ZoneMinder / mlapi

An easy to use/extend object recognition API you can locally install. Python+Flask. Also works with ZMES!
Other
58 stars 34 forks source link

Error running model: invalid index to scalar variable. #44

Closed juan11perez closed 3 years ago

juan11perez commented 3 years ago

Running mlapi 2.2.23 with opencv 4.5.4. Following error on object detection:

`Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Setting up signal handler for logs]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Switching global logger to ZMLog]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Switched to ZM logs]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Reading config from: /var/lib/zmeventnotification/mlapiconfig.ini]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Reading config from: /var/lib/zmeventnotification/mlapiconfig.ini]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Reading secrets from: /etc/zm/secrets.ini]

Nov 3 13:25:41 UNRAID web_php[813]: 11/03/21 13:25:41.182655 web_php[813].INF [172.10.0.1] [Login successful for user "juan"] at /usr/share/zoneminder/www/api/app/Controller/AppController.php line 86

Nov 3 13:25:41 UNRAID web_php[813]: 11/03/21 13:25:41.185551 web_php[813].INF [172.10.0.1] [Creating token for "juan"] at /usr/share/zoneminder/www/api/app/Controller/HostController.php line 157

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [--------| mlapi version:2.2.23, pyzm version:0.3.55 |--------]

Nov 3 13:25:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Using bjoern as WSGI server]

Nov 3 13:26:41 UNRAID /mlapi.py[1219]: ERR [zm_mlapi] [Error running model: invalid index to scalar variable.]

Nov 3 13:26:41 UNRAID /mlapi.py[1219]: INF [zm_mlapi] [Removing matches to past detections for monitor:6]`

If YoloV4 is set to 'enabled': 'no' in mlapiconfig.ini error disapears.

This error is not present with opencv 4.5.3

pliablepixels commented 3 years ago

It looks like the API output changed in OpenCV 4.5.4 See https://github.com/opencv/opencv/issues/20923

In previous versions, the dimensions were returned as an array of array but now its just an array. I'm not sure if this compatibility issue will be fixed, or if this is the new format, but for now, see temporary fix at https://forums.zoneminder.com/viewtopic.php?f=43&t=31129&p=123755&hilit=scalar#p123755

juan11perez commented 3 years ago

thank you very much for your prompt reply. That workaround indeed removes the error. should i close this? or would you like to keep it open?

pliablepixels commented 3 years ago

Let's keep it open till I fix it correctly

pliablepixels commented 3 years ago

Could you replace your yolo.py with this one and test? https://github.com/pliablepixels/pyzm/blob/5dfd2b7bfe57024dabc125c53c10b2c9f8e1a82b/pyzm/ml/yolo.py

I am not running 4.5.4 at the moment

juan11perez commented 3 years ago

@pliablepixels tested your new yolo.py. Works fine, no errors, object recognition as expected. i turned TPU off to ensure yolo processing and it works fine.

baudneo commented 3 years ago

Good job on finding which exact version!

thekazak commented 2 years ago

Same error "IndexError: invalid index to scalar variable" with latest opencv/opencv-contrib 4.6.0-dev, compiled from source

08/11/22 20:42:06 zmesdetect_m1[4359] INF ZMLog.py:292 [Setting up signal handler for logs] 08/11/22 20:42:06 zmesdetect_m1[4359] INF ZMLog.py:301 [Switching global logger to ZMLog] 08/11/22 20:42:06 zmesdetect_m1[4359] INF zm_detect.py:284 [---------| app:6.1.28, pyzm:0.3.56, ES:6.1.28 , OpenCV:4.6.0-dev|------------] 08/11/22 20:42:06 zmesdetect_m1[4359] INF utils.py:405 [Reading config from: /etc/zm/objectconfig.ini] 08/11/22 20:42:06 zmesdetect_m1[4359] INF utils.py:410 [Reading secrets from: /etc/zm/secrets.ini] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:382 [Secret token found in config: !ZM_PORTAL] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:382 [Secret token found in config: !ZM_USER] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:382 [Secret token found in config: !ZM_PASSWORD] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:382 [Secret token found in config: !ZM_API_PORTAL] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:382 [Secret token found in config: !ML_USER] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:382 [Secret token found in config: !ML_PASSWORD] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:382 [Secret token found in config: !PLATEREC_ALPR_KEY] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG1 utils.py:445 [allowing self-signed certs to work...] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:455 [Now checking for monitor overrides] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG3 utils.py:473 [[monitor-1] overrides key:wait with value:5] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:174 [import_zm_zones: match_reason=True and reason=Motion] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG2 utils.py:177 [Getting ZM zones using https://zm.webtocloud.net:44313/zm/api/zones/forMonitor/1.json?username=xxx&password=yyy&user=xxx&pass=yyy] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG1 utils.py:225 [dropping public_entrance as zones in alarm cause is Motion] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG1 utils.py:225 [dropping private_entrance as zones in alarm cause is Motion] 08/11/22 20:42:06 zmesdetect_m1[4359] DBG3 utils.py:522 [Finally, doing parameter substitution] 08/11/22 20:42:06 zmesdetect_m1[4359] INF zm_detect.py:309 [Importing local classes for Object/Face] 08/11/22 20:42:07 zmesdetect_m1[4359] INF zm_detect.py:334 [Connecting with ZM APIs] 08/11/22 20:42:07 zmesdetect_m1[4359] DBG2 api.py:72 [API SSL certificate check has been disbled] 08/11/22 20:42:07 zmesdetect_m1[4359] DBG1 api.py:181 [using username/password for login] 08/11/22 20:42:07 zmesdetect_m1[4359] DBG2 api.py:210 [Using new token API] 08/11/22 20:42:07 zmesdetect_m1[4359] DBG1 api.py:217 [Access token expires on:2022-08-11 22:42:07.248078 [7200s]] 08/11/22 20:42:07 zmesdetect_m1[4359] DBG1 api.py:221 [Refresh token expires on:2022-08-12 20:42:07.248280 [86400s]] 08/11/22 20:42:07 zmesdetect_m1[4359] DBG2 zm_detect.py:342 [using ml_sequence] 08/11/22 20:42:07 zmesdetect_m1[4359] DBG2 zm_detect.py:354 [using stream_sequence] 08/11/22 20:42:07 zmesdetect_m1[4359] INF zm_detect.py:418 [Sleeping for 5 seconds before inferencing] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:160 [Resetting models, will be loaded on next run] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 detect_sequence.py:637 [Using manual locking as we are only using one model] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 Media.py:51 [Media get SSL certificate check has been disbled] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 Media.py:99 [Using URL 50156 for stream] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 Media.py:114 [We will only process frames: ['snapshot', 'alarm']] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 Media.py:137 [No need to start streams, we are picking images from https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50156] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 Media.py:271 [Reading https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50156&fid=snapshot] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 api.py:144 [No need to relogin as access token still has 119.91638281666665 minutes remaining] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 api.py:272 [make_request called with url=https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50156&fid=snapshot payload={} type=get query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9. eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjYwMjM5NzI3LCJleHAiOjE2NjAyNDY5MjcsInVzZXIiOiJhZG1pbiIsInR5cGUiOiJhY2Nlc3MifQ.GIt_mrSQB-pa1GVThyV5rG_KESJ3ROjM5Tm6PhlYIWQ'}] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:654 [perf: Starting for frame:snapshot] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:664 [Sequence of detection types to execute: ['object']] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:669 [============ Frame: snapshot Running object detection type in sequence ==================] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 detect_sequence.py:174 [Skipping TPU object detection as it is disabled] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 detect_sequence.py:178 [Loading sequence: YoloV4 GPU/CPU] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 detect_sequence.py:179 [Initializing model type:object with options:{'name': 'YoloV4 GPU/CPU', 'enabled': 'yes', 'object_config': '/var/lib/zmeventnotification/models/yolov4/yolov4.cfg', 'object_weights': '/var/lib/zmeventnotification/models/yolov4/yolov4.weights', 'object_labels': '/var/lib/zmeventnotification/models/yolov4/coco.names', 'object_min_confidence': 0.3, 'object_framework': 'opencv', 'object_processor': 'gpu', 'gpu_max_processes': 1, 'gpu_max_lock_wait': 100, 'cpu_max_processes': 3, 'cpu_max_lock_wait': 100, 'max_detection_size': '90%', 'disable_locks': 'no'}] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:38 [portalock: max:1, name:pyzm_uid33_gpu_lock, timeout:100] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:50 [Waiting for pyzm_uid33_gpu_lock portalock...] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:52 [Got pyzm_uid33_gpu_lock portalock] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 detect_sequence.py:689 [object has a same_model_sequence strategy of first] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:701 [--------- Frame:snapshot Running variation: #1 -------------] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:136 [detect extracted image dimensions as: 800wx1422h] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:47 [pyzm_uid33_gpu_lock portalock already acquired] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 yolo.py:85 [|--------- Loading "YoloV4 GPU/CPU" model from disk -------------|] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 yolo.py:100 [perf: processor:gpu Yolo initialization (loading /var/lib/zmeventnotification/models/yolov4/yolov4.weights model from disk) took: 69.05 ms] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:115 [Setting CUDA backend for OpenCV] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 yolo.py:116 [If you did not set your CUDA_ARCH_BIN correctly during OpenCV compilation, you will get errors during detection related to invalid device/make_policy] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 yolo.py:160 [|---------- YOLO (input image: 800w1422h, model resize dimensions: 416w416h) ----------|] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:68 [Released pyzm_uid33_gpu_lock portalock] 08/11/22 20:42:12 zmesdetect_m1[4359] ERR detect_sequence.py:713 [Error running model: invalid index to scalar variable.] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 detect_sequence.py:714 [Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/detect_sequence.py", line 710, in detect_stream _b,_l,_c,_m = m.detect(image=frame) File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/object.py", line 65, in detect b,l,c,_model_names = self.model.detect(image) File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/yolo.py", line 168, in detect ln = self.get_output_layers() File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/yolo.py", line 129, in get_output_layers output_layers = [ File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/yolo.py", line 130, in layer_names[i[0] - 1] for i in self.net.getUnconnectedOutLayers() IndexError: invalid index to scalar variable. ] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 detect_sequence.py:770 [We did not find any object matches in frame: snapshot] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 Media.py:271 [Reading https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50156&fid=alarm] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 api.py:144 [No need to relogin as access token still has 119.91260125000001 minutes remaining] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 api.py:272 [make_request called with url=https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50156&fid=alarm payload={} type=get query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjYwMjM5NzI3LCJleHAiOjE2NjAyNDY5MjcsInVzZXIiOiJhZG1pbiIsInR5cGUiOiJhY2Nlc3MifQ.GIt_mrSQB-pa1GVThyV5rG_KESJ3ROjM5Tm6PhlYIWQ'}] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:654 [perf: Starting for frame:alarm] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:664 [Sequence of detection types to execute: ['object']] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:669 [============ Frame: alarm Running object detection type in sequence ==================] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG3 detect_sequence.py:689 [object has a same_model_sequence strategy of first] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:701 [--------- Frame:alarm Running variation: #1 -------------] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:136 [detect extracted image dimensions as: 800wx1422h] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:50 [Waiting for pyzm_uid33_gpu_lock portalock...] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:52 [Got pyzm_uid33_gpu_lock portalock] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 yolo.py:160 [|---------- YOLO (input image: 800w1422h, model resize dimensions: 416w416h) ----------|] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:68 [Released pyzm_uid33_gpu_lock portalock] 08/11/22 20:42:12 zmesdetect_m1[4359] ERR detect_sequence.py:713 [Error running model: invalid index to scalar variable.] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 detect_sequence.py:714 [Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/detect_sequence.py", line 710, in detect_stream _b,_l,_c,_m = m.detect(image=frame) File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/object.py", line 65, in detect b,l,c,_model_names = self.model.detect(image) File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/yolo.py", line 168, in detect ln = self.get_output_layers() File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/yolo.py", line 129, in get_output_layers output_layers = [ File "/usr/local/lib/python3.9/dist-packages/pyzm/ml/yolo.py", line 130, in layer_names[i[0] - 1] for i in self.net.getUnconnectedOutLayers() IndexError: invalid index to scalar variable. ] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 detect_sequence.py:770 [We did not find any object matches in frame: alarm] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG2 yolo.py:64 [pyzm_uid33_gpu_lock portalock already released] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 detect_sequence.py:827 [perf: TOTAL detection sequence (with image loads) took: 306.11 ms to process 50156] 08/11/22 20:42:12 zmesdetect_m1[4359] DBG1 zm_detect.py:557 [Closing logs]

thekazak commented 2 years ago

The error shows up with opencv/contrib 4.6.0-dev only There is no problem with opencv/contrib 4.5.5

baudneo commented 2 years ago

I'll post a link to the fix and how to apply it here shortly

baudneo commented 2 years ago

See this issue thread and apply the same fix

https://github.com/ZoneMinder/zmeventnotification/issues/416

thekazak commented 2 years ago

After applying fix in ZoneMinder/zmeventnotification#416 now error "IndexError: invalid index to scalar variable" disapeared with opencv-contrib 4.6.0

But I experience now object_detect does not match anything. Not sure what it came from, but I tried delete /var/lib/zmeventnotification and install ES again by install.sh . It used to match/detect person pretty well before updated opencv 4.6.0

Is there anything alse to be reset ?

08/12/22 20:04:46 zmesdetect_m1[33801] INF ZMLog.py:292 [Setting up signal handler for logs] 08/12/22 20:04:46 zmesdetect_m1[33801] INF ZMLog.py:301 [Switching global logger to ZMLog] 08/12/22 20:04:46 zmesdetect_m1[33801] INF zm_detect.py:284 [---------| app:6.1.28, pyzm:0.3.56, ES:6.1.28 , OpenCV:4.6.0-dev|------------] 08/12/22 20:04:46 zmesdetect_m1[33801] INF utils.py:405 [Reading config from: /etc/zm/objectconfig.ini] 08/12/22 20:04:46 zmesdetect_m1[33801] INF utils.py:410 [Reading secrets from: /etc/zm/secrets.ini] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG2 utils.py:382 [Secret token found in config: !ZM_PORTAL] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG2 utils.py:382 [Secret token found in config: !ZM_USER] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG2 utils.py:382 [Secret token found in config: !ZM_PASSWORD] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG2 utils.py:382 [Secret token found in config: !ZM_API_PORTAL] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG2 utils.py:382 [Secret token found in config: !ML_USER] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG2 utils.py:382 [Secret token found in config: !ML_PASSWORD] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG2 utils.py:382 [Secret token found in config: !PLATEREC_ALPR_KEY] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG1 utils.py:445 [allowing self-signed certs to work...] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG2 utils.py:455 [Now checking for monitor overrides] 08/12/22 20:04:46 zmesdetect_m1[33801] DBG3 utils.py:522 [Finally, doing parameter substitution] 08/12/22 20:04:46 zmesdetect_m1[33801] INF zm_detect.py:309 [Importing local classes for Object/Face] 08/12/22 20:04:47 zmesdetect_m1[33801] INF zm_detect.py:334 [Connecting with ZM APIs] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 api.py:72 [API SSL certificate check has been disbled] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 api.py:181 [using username/password for login] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 api.py:210 [Using new token API] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 api.py:217 [Access token expires on:2022-08-12 22:04:47.369973 [7200s]] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 api.py:221 [Refresh token expires on:2022-08-13 20:04:47.370239 [86400s]] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 zm_detect.py:342 [using ml_sequence] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 zm_detect.py:354 [using stream_sequence] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 detect_sequence.py:160 [Resetting models, will be loaded on next run] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG3 detect_sequence.py:634 [Using automatic locking as we are switching between models] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 Media.py:51 [Media get SSL certificate check has been disbled] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 Media.py:99 [Using URL 50365 for stream] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 Media.py:114 [We will only process frames: ['snapshot', 'alarm']] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 Media.py:137 [No need to start streams, we are picking images from https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50365] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG3 Media.py:271 [Reading https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50365&fid=snapshot] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG3 api.py:144 [No need to relogin as access token still has 119.99991481666666 minutes remaining] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG3 api.py:272 [make_request called with url=https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50365&fid=snapshot payload={} type=get query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjYwMzIzODg3LCJleHAiOjE2NjAzMzEwODcsInVzZXIiOiJhZG1pbiIsInR5cGUiOiJhY2Nlc3MifQ.gKNSwpiE3Ei6-kSiz1L8mxUvx_GjRt24k8VB3JXw8O4'}] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 detect_sequence.py:654 [perf: Starting for frame:snapshot] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 detect_sequence.py:664 [Sequence of detection types to execute: ['object', 'face']] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 detect_sequence.py:669 [============ Frame: snapshot Running object detection type in sequence ==================] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 detect_sequence.py:174 [Skipping TPU object detection as it is disabled] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 detect_sequence.py:178 [Loading sequence: YoloV4 GPU/CPU] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 detect_sequence.py:179 [Initializing model type:object with options:{'name': 'YoloV4 GPU/CPU', 'enabled': 'yes', 'object_config': '/var/lib/zmeventnotification/models/yolov4/yolov4.cfg', 'object_weights': '/var/lib/zmeventnotification/models/yolov4/yolov4.weights', 'object_labels': '/var/lib/zmeventnotification/models/yolov4/coco.names', 'object_min_confidence': 0.3, 'object_framework': 'opencv', 'object_processor': 'gpu', 'gpu_max_processes': 1, 'gpu_max_lock_wait': 100, 'cpu_max_processes': 3, 'cpu_max_lock_wait': 100, 'max_detection_size': '90%', 'disable_locks': 'no'}] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 yolo.py:54 [portalock: max:1, name:pyzm_uid33_gpu_lock, timeout:100] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG3 detect_sequence.py:689 [object has a same_model_sequence strategy of first] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 detect_sequence.py:701 [--------- Frame:snapshot Running variation: #1 -------------] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 yolo.py:152 [detect extracted image dimensions as: 800wx1422h] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 yolo.py:68 [Waiting for pyzm_uid33_gpu_lock portalock...] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 yolo.py:70 [Got pyzm_uid33_gpu_lock portalock] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 yolo.py:102 [|--------- Loading "YoloV4 GPU/CPU" model from disk -------------|] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 yolo.py:113 [You are using OpenCV >= 4.5.4, making sure we fix getUnconnectedOutLayers() API] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 yolo.py:115 [perf: processor:gpu Yolo initialization (loading /var/lib/zmeventnotification/models/yolov4/yolov4.weights model from disk) took: 251.88 ms] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG2 yolo.py:130 [Setting CUDA backend for OpenCV] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG3 yolo.py:131 [If you did not set your CUDA_ARCH_BIN correctly during OpenCV compilation, you will get errors during detection related to invalid device/make_policy] 08/12/22 20:04:47 zmesdetect_m1[33801] DBG1 yolo.py:175 [|---------- YOLO (input image: 800w1422h, model resize dimensions: 416w416h) ----------|] 08/12/22 20:04:48 zmesdetect_m1[33801] DBG2 yolo.py:86 [Released pyzm_uid33_gpu_lock portalock] 08/12/22 20:04:48 zmesdetect_m1[33801] DBG1 yolo.py:199 [perf: processor:gpu Yolo detection took: 1274.97 ms] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 yolo.py:233 [perf: processor:gpu Yolo NMS filtering took: 0.78 ms] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 object.py:66 [core model detection over, got 0 objects. Now filtering] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 object.py:70 [Max object size found to be: 90%] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 object.py:78 [Converted 90% to 1023840.0] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 object.py:103 [Returning filtered list of 0 objects.] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 detect_sequence.py:711 [This model iteration inside object found: labels: [],conf:[]] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 detect_sequence.py:424 [Max object size found to be: 90%] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 detect_sequence.py:432 [Converted 90% to 1023840.0] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 detect_sequence.py:448 [No polygons, adding full image polygon: {'name': 'full_image', 'value': [(0, 0), (1080, 0), (1080, 1920), (0, 1920)], 'pattern': None}] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG3 detect_sequence.py:228 [resized polygons x=0.7407407407407407/y=0.740625: [{'name': 'full_image', 'value': [(0, 0), (800, 0), (800, 1422), (0, 1422)], 'pattern': None}]] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 detect_sequence.py:770 [We did not find any object matches in frame: snapshot] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG1 detect_sequence.py:669 [============ Frame: snapshot Running face detection type in sequence ==================] 08/12/22 20:04:49 zmesdetect_m1[33801] DBG2 detect_sequence.py:190 [Skipping TPU face detection as it is disabled] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG1 face_dlib.py:42 [perf: processor:cpu Face Recognition library load time took: 0.00 ms ] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG1 face_dlib.py:50 [Initializing face recognition with model:cnn upsample:1, jitters:0] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG2 face_dlib.py:73 [portalock: max:3, name:pyzm_uid33_cpu_lock, timeout:100] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG1 face_dlib.py:97 [trained file not found, reading from images and doing training...] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG1 face_dlib.py:100 [If you are using a GPU and run out of memory, do the training using zm_train_faces.py. In this case, other models like yolo may already take up a lot of GPU memory] 08/12/22 20:04:50 zmesdetect_m1[33801] ERR face_train_dlib.py:135 [No known faces found to train, encoding file not created] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG1 face_train_dlib.py:152 [perf: Face Recognition training took: 9.64 ms] 08/12/22 20:04:50 zmesdetect_m1[33801] ERR face_dlib.py:110 [Error loading KNN model: [Errno 2] No such file or directory: '/var/lib/zmeventnotification/known_faces/faces.dat'] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG3 detect_sequence.py:689 [face has a same_model_sequence strategy of union] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG1 detect_sequence.py:701 [--------- Frame:snapshot Running variation: #1 -------------] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG1 face_dlib.py:164 [|---------- Dlib Face recognition (input image: 800w1422h) ----------|] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG3 face_dlib.py:174 [Face options={'name': 'DLIB based face recognition', 'enabled': 'yes', 'save_unknown_faces': 'yes', 'save_unknown_faces_leeway_pixels': 100, 'face_detection_framework': 'dlib', 'known_images_path': '/var/lib/zmeventnotification/known_faces', 'unknown_images_path': '/var/lib/zmeventnotification/unknown_faces', 'face_model': 'cnn', 'face_train_model': 'cnn', 'face_recog_dist_threshold': '0.6', 'face_num_jitters': '1', 'face_upsample_times': '1', 'gpu_max_processes': 1, 'gpu_max_lock_wait': 100, 'cpu_max_processes': 3, 'cpu_max_lock_wait': 100, 'max_size': 800, 'disable_locks': 'no'}] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG2 face_dlib.py:123 [Waiting for pyzm_uid33_cpu_lock portalock...] 08/12/22 20:04:50 zmesdetect_m1[33801] DBG2 face_dlib.py:125 [Got pyzm_uid33_cpu_lock lock...] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG1 face_dlib.py:206 [perf: processor:cpu Finding faces took 19282.68 ms] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG1 face_dlib.py:141 [Released pyzm_uid33_cpu_lock portalock] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG1 face_dlib.py:218 [perf: processor:cpu Computing face recognition distances took 0.97 ms] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG2 detect_sequence.py:711 [This model iteration inside face found: labels: [],conf:[]] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG2 detect_sequence.py:424 [Max object size found to be: 90%] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG2 detect_sequence.py:432 [Converted 90% to 1023840.0] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG2 detect_sequence.py:770 [We did not find any face matches in frame: snapshot] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG3 Media.py:271 [Reading https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50365&fid=alarm] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG3 api.py:144 [No need to relogin as access token still has 119.62694315 minutes remaining] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG3 api.py:272 [make_request called with url=https://zm.webtocloud.net:44313/zm/index.php?view=image&eid=50365&fid=alarm payload={} type=get query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjYwMzIzODg3LCJleHAiOjE2NjAzMzEwODcsInVzZXIiOiJhZG1pbiIsInR5cGUiOiJhY2Nlc3MifQ.gKNSwpiE3Ei6-kSiz1L8mxUvx_GjRt24k8VB3JXw8O4'}] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG1 detect_sequence.py:654 [perf: Starting for frame:alarm] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG1 detect_sequence.py:664 [Sequence of detection types to execute: ['object', 'face']] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG1 detect_sequence.py:669 [============ Frame: alarm Running object detection type in sequence ==================] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG3 detect_sequence.py:689 [object has a same_model_sequence strategy of first] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG1 detect_sequence.py:701 [--------- Frame:alarm Running variation: #1 -------------] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG2 yolo.py:152 [detect extracted image dimensions as: 800wx1422h] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG2 yolo.py:68 [Waiting for pyzm_uid33_gpu_lock portalock...] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG2 yolo.py:70 [Got pyzm_uid33_gpu_lock portalock] 08/12/22 20:05:09 zmesdetect_m1[33801] DBG1 yolo.py:175 [|---------- YOLO (input image: 800w1422h, model resize dimensions: 416w416h) ----------|] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 yolo.py:86 [Released pyzm_uid33_gpu_lock portalock] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG1 yolo.py:199 [perf: processor:gpu Yolo detection took: 2145.69 ms] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 yolo.py:233 [perf: processor:gpu Yolo NMS filtering took: 1.05 ms] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 object.py:66 [core model detection over, got 0 objects. Now filtering] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 object.py:70 [Max object size found to be: 90%] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 object.py:78 [Converted 90% to 1023840.0] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 object.py:103 [Returning filtered list of 0 objects.] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 detect_sequence.py:711 [This model iteration inside object found: labels: [],conf:[]] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 detect_sequence.py:424 [Max object size found to be: 90%] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 detect_sequence.py:432 [Converted 90% to 1023840.0] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 detect_sequence.py:770 [We did not find any object matches in frame: alarm] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG1 detect_sequence.py:669 [============ Frame: alarm Running face detection type in sequence ==================] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG3 detect_sequence.py:689 [face has a same_model_sequence strategy of union] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG1 detect_sequence.py:701 [--------- Frame:alarm Running variation: #1 -------------] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG1 face_dlib.py:164 [|---------- Dlib Face recognition (input image: 800w1422h) ----------|] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG3 face_dlib.py:174 [Face options={'name': 'DLIB based face recognition', 'enabled': 'yes', 'save_unknown_faces': 'yes', 'save_unknown_faces_leeway_pixels': 100, 'face_detection_framework': 'dlib', 'known_images_path': '/var/lib/zmeventnotification/known_faces', 'unknown_images_path': '/var/lib/zmeventnotification/unknown_faces', 'face_model': 'cnn', 'face_train_model': 'cnn', 'face_recog_dist_threshold': '0.6', 'face_num_jitters': '1', 'face_upsample_times': '1', 'gpu_max_processes': 1, 'gpu_max_lock_wait': 100, 'cpu_max_processes': 3, 'cpu_max_lock_wait': 100, 'max_size': 800, 'disable_locks': 'no'}] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 face_dlib.py:123 [Waiting for pyzm_uid33_cpu_lock portalock...] 08/12/22 20:05:12 zmesdetect_m1[33801] DBG2 face_dlib.py:125 [Got pyzm_uid33_cpu_lock lock...] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG1 face_dlib.py:206 [perf: processor:cpu Finding faces took 8613.39 ms] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG1 face_dlib.py:141 [Released pyzm_uid33_cpu_lock portalock] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG1 face_dlib.py:218 [perf: processor:cpu Computing face recognition distances took 0.28 ms] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG2 detect_sequence.py:711 [This model iteration inside face found: labels: [],conf:[]] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG2 detect_sequence.py:424 [Max object size found to be: 90%] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG2 detect_sequence.py:432 [Converted 90% to 1023840.0] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG2 detect_sequence.py:770 [We did not find any face matches in frame: alarm] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG1 detect_sequence.py:827 [perf: TOTAL detection sequence (with image loads) took: 33440.35 ms to process 50365] 08/12/22 20:05:20 zmesdetect_m1[33801] DBG1 zm_detect.py:557 [Closing logs]

baudneo commented 2 years ago

The only thing changed in the fix is how the opencv version is processed so there really shouldn't be any issues.

Are you testing with an event that had previously detected objects?

thekazak commented 2 years ago

I will test more thorougly Just wanted to know do I need to reset/cleanup anything else in addition to /var/lib/zmeventnotification and python3 bindings zmes_hook_helpers and pyzm , in my case located in /usr/local/lib/python3.9/dist-packages ?

baudneo commented 2 years ago

No, replacing yolo.py will not affect anything else and you will not need to change anything else. It's a rolling hot fix that shouldn't do anything but take care of the index error you were seeing before.

thekazak commented 2 years ago

Seems issue with detected objects is somehow connected with ZM only. Fixed yolo.py is working just fine with opencv-contrib 4.6.0

Helipil0t commented 1 year ago

So the fixed yolo.py is no longer working with ZoneMinder 1.36.32. OpenCV 4.6.0-dev Python 3.10.6. I had to remove a few lines from the code to get things working. I posted more details here: https://forums.zoneminder.com/viewtopic.php?t=32223

baudneo commented 1 year ago

By the fixed yolo.py do you mean the one I uploaded or do you mean the the one using/not using the indices based on opencv version?