search

ZoneMinder / mlapi

An easy to use/extend object recognition API you can locally install. Python+Flask. Also works with ZMES!
Other
58 stars 35 forks source link

mlapi does not behave as ES does #36

Closed mfauvain closed 3 years ago

mfauvain commented 3 years ago

I have different behaviour from mlapi than from the ES server directly. Running a test on an event reported by mlapi I have the following : [s] detected:person,car--SPLIT--{"labels": ["person", "car"], "boxes": [[156, 251, 288, 515], [203, 3, 441, 93]], "frame_id": "snapshot", "confidences": [0.9882749915122986, 0.8627339005470276], "image_dimensions": {"original": [480, 640], "resized": [600, 800]}}

and log from zmesdetect shows : zmesdetect_m10[65922] INF zm_detect.py:327 [Connecting with ZM APIs] 02/26/21 18:24:14 zmesdetect_m10[65922] INF zm_detect.py:48 [Detecting using remote API Gateway http://192.168.1.178:5002/api/v1] 02/26/21 18:24:17 zmesdetect_m10[65922] INF zm_detect.py:495 [Prediction string:[s] detected:person,car]

That matches what mlapi is sending : [DBG 3] full_image intersects object:person[[(22, 390), (240, 390), (240, 584), (22, 584)]] Feb 26 2021 18:27:16.582004 [DBG 3] breaking out of same model loop, as matches found and strategy is "first" Feb 26 2021 18:27:16.582369 [DBG 2] pyzm_uid1000_cpu_lock portalock already released Feb 26 2021 18:27:16.582394 [DBG 1] perf: TOTAL detection sequence (with image loads) took: 2074.66 ms to process 622971 Feb 26 2021 18:27:16.582806 [DBG 1] Returning {'matched_data': {'boxes': [[156, 251, 288, 515], [203, 3, 441, 93]], 'error_boxes': [], 'labels': ['person', 'car'], 'confidences': [0.9882749915122986, 0.8627339005470276], 'frame_id': 'snapshot', 'image_dimensions': {'original': (480, 640), 'resized': (600, 800)}, 'image': None}, 'all_matches': [{'frame_id': 'snapshot', 'boxes': [[156, 251, 288, 515], [203, 3, 441, 93]], 'error_boxes': [], 'labels': ['person', 'car'], 'confidences': [0.9882749915122986, 0.8627339005470276], 'models': ['object']}, {'frame_id': 'alarm', 'boxes': [[203, 3, 453, 93], [22, 390, 240, 584]], 'error_boxes': [], 'labels': ['car', 'person'], 'confidences': [0.7241158485412598, 0.6331931948661804], 'models': ['object']}], 'polygons': []}

However, this is not the desired behaviour, nothing (car or person) should be reported. And when run directly on the ES server (not using mlapi) the output is : zmesdetect_m10[66169] INF zm_detect.py:279 [---------| app:6.1.12, pyzm:0.3.25, ES:6.1.12 , OpenCV:4.5.1|------------] 02/26/21 18:29:59 zmesdetect_m10[66169] INF zm_detect.py:304 [Importing local classes for Object/Face] 02/26/21 18:29:59 zmesdetect_m10[66169] INF zm_detect.py:327 [Connecting with ZM APIs] 02/26/21 18:30:01 zmesdetect_m10[66169] INF detect_sequence.py:282 [object:car at POLYGON ((162 3, 364 3, 364 75, 162 75, 162 3)) does not fall into any polygons, removing...] which is the correct beahviour. 'car' is found but not inside the defined polygon, so nothing reported.

My monitors are defined in objectconfig.ini on the ES server. The specific monitor is defined like so : [monitor-10] resize=no detection_sequence=object smaller_m10=535,64 597,57 543,474 362,476 smaller_m10_zone_detection_pattern=(person|bear|dog|cat|car|bicycle|motorbike|truck) all_zone_detection_pattern=(car|bicycle|motorbike|truck)

It seems it is not 'correctly' passed on to mlapi. I tried to set the exact same monitor definition in mlapiconfig.ini on the mlapi server, but it throws an error analyzing the same event : Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__ return self.wsgi_app(environ, start_response) File "/usr/lib/python3.9/site-packages/flask/app.py", line 2450, in wsgi_app response = self.handle_exception(e) File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router return original_handler(e) File "/usr/lib/python3.9/site-packages/flask/app.py", line 1867, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python3.9/site-packages/flask/_compat.py", line 38, in reraise raise value.with_traceback(tb) File "/usr/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router return original_handler(e) File "/usr/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python3.9/site-packages/flask/_compat.py", line 38, in reraise raise value.with_traceback(tb) File "/usr/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 468, in wrapper resp = resource(*args, **kwargs) File "/usr/lib/python3.9/site-packages/flask/views.py", line 89, in view return self.dispatch_request(*args, **kwargs) File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 583, in dispatch_request resp = meth(*args, **kwargs) File "/usr/lib/python3.9/site-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper return fn(*args, **kwargs) File "/home/x/.mlapi/mlapi.py", line 229, in post matched_data,all_matches = m.detect_stream(stream=stream, options=stream_options, ml_overrides=ml_overrides) File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 466, in detect_stream _b,_l,_c, _e = self._filter_patterns(seq,_b,_l,_c, polygons) File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 246, in _filter_patterns polygons[:] = self._rescale_polygons(polygons, neww / oldw, newh / oldh) File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 199, in _rescale_polygons for x, y in p['value']: KeyError: 'value' I've re-installed mlapi server again, no better. I don't understand why, when leaving mlapiconfig.ini with no monitor definition it 'works' in the sense that it does not crash (albeit reporting an event when it should not) and when setting the monitor there, it crashes.. Can you help?

pliablepixels commented 3 years ago

Thanks for testing. Could you try mlapi master, and run it manually with --debug ?

Then:

  1. Please post the data it prints at the start.
  2. Data it prints for one detection

Suggestion: When posting code, please put your logs in three backticks so it is easier to read. See https://docs.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks

mfauvain commented 3 years ago

Running master with --debug I have (with monitors defined in objectconfig.ini only)

python3 /home/x/.mlapi/mlapi.py -c mlapiconfig.ini --debug
Feb 27 2021 11:11:29.590252 [DBG 1] secret filename: ./secrets.ini
Feb 27 2021 11:11:29.590760 [DBG 1] Secret token found in config: !MLAPI_SECRET_KEY
Feb 27 2021 11:11:29.590950 [DBG 1] Secret token found in config: !PLATEREC_ALPR_KEY
Feb 27 2021 11:11:29.591025 [DBG 4] Doing parameter substitution for globals
Feb 27 2021 11:11:29.591267 [DBG 4] Doing parameter substitution for monitor specific entities
Feb 27 2021 11:11:29.593220 [DBG 1] Opening DB at ./db/db.json
Feb 27 2021 11:11:29.593353 [DBG 1] DB engine ready
Feb 27 2021 11:11:29.595602 [DBG 2] API SSL certificate check has been disbled
Feb 27 2021 11:11:29.595656 [DBG 1] using username/password for login
Feb 27 2021 11:11:29.708032 [DBG 2] Using new token API
Feb 27 2021 11:11:29.708213 [DBG 1] Access token expires on:2021-02-27 13:11:29.708151 [7200s]
Feb 27 2021 11:11:29.708356 [DBG 1] Refresh token expires on:2021-02-28 11:11:29.708309 [86400s]
Feb 27 2021 11:11:29.708470 [DBG 3] No need to relogin as access token still has 119.99999516666666 minutes remaining
Feb 27 2021 11:11:29.708586 [DBG 4] make_request called with url=http://192.168.1.189:8095/api/host/gettimezone.json payload={} type=get query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjE0NDQ1ODg5LCJleHAiOjE2MTQ0NTMwODksInVzZXIiOiJhZG1pbiIsInR5cGUiOiJhY2Nlc3MifQ.V4OxNT45z8DCRkCCOuTYjlbOSrLAFVdPvNw7tkayTcw'}
Feb 27 2021 11:11:29.744810 [DBG 2] mapping legacy ml data from config
Feb 27 2021 11:11:29.744959 [DBG 3] Changing detection_mode from all to most_models to adapt to new features
Feb 27 2021 11:11:29.745113 [DBG 1] Resetting models, will be loaded on next run
Feb 27 2021 11:11:29.745214 [INF] --------| mlapi version:2.2.3, pyzm version:0.3.35 |--------
Feb 27 2021 11:11:29.745286 [INF] Using bjoern as WSGI server

and sending in a detection I have :

Feb 27 2021 11:15:59.318295 [DBG 1] Monitor ID not specified, or not found in mlapi config, using zm_detect overrides
Feb 27 2021 11:15:59.318395 [DBG 2] mapping legacy ml data from config
Feb 27 2021 11:15:59.318503 [DBG 1] Object Recognition requested
Feb 27 2021 11:15:59.318643 [DBG 1] Calling detect streams with 622971 and {'resize': None, 'strategy': 'first', 'frame_set': 'alarm,snapshot', 'disable_ssl_cert_check': True, 'api': <pyzm.api.ZMApi object at 0x7fde4fb271c0>, 'polygons': [], 'monitorid': '10'} and ml_overrides={'model_sequence': 'object', 'object': {'pattern': '(person|bear|dog|cat|car|bicycle|motorbike|truck)'}, 'face': {'pattern': None}, 'alpr': {'pattern': None}} ml_options={'object': {'general': {'pattern': '.*', 'disable_locks': 'no', 'same_model_sequence_strategy': 'first'}, 'sequence': [{'tpu_max_processes': 1, 'tpu_max_lock_wait': 120, 'gpu_max_processes': 1, 'gpu_max_lock_wait': 120, 'cpu_max_processes': 3, 'cpu_max_lock_wait': 120, 'max_detection_size': '90%', 'object_config': 'models/yolov3/yolov3.cfg', 'object_weights': 'models/yolov3/yolov3.weights', 'object_labels': 'models/yolov3/coco.names', 'object_min_confidence': 0.4, 'object_framework': 'opencv', 'object_processor': 'cpu'}]}, 'general': {'model_sequence': 'object'}}
Feb 27 2021 11:15:59.318720 [DBG 3] Using manual locking as we are only using one model
Feb 27 2021 11:15:59.319267 [DBG 2] Media get SSL certificate check has been disbled
Feb 27 2021 11:15:59.319410 [DBG 2] Using URL 622971 for stream
Feb 27 2021 11:15:59.319480 [DBG 2] We will only process frames: ['alarm', 'snapshot']
Feb 27 2021 11:15:59.319529 [DBG 2] No need to start streams, we are picking images from http://192.168.1.189:8095/index.php?view=image&eid=622971
Feb 27 2021 11:15:59.319597 [DBG 3] Reading http://192.168.1.189:8095/index.php?view=image&eid=622971&fid=alarm
Feb 27 2021 11:15:59.319665 [DBG 3] No need to relogin as access token still has 119.88258429999999 minutes remaining
Feb 27 2021 11:15:59.319730 [DBG 4] make_request called with url=http://192.168.1.189:8095/index.php?view=image&eid=622971&fid=alarm payload={} type=get query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjE0NDQ2MTUyLCJleHAiOjE2MTQ0NTMzNTIsInVzZXIiOiJhZG1pbiIsInR5cGUiOiJhY2Nlc3MifQ.N-POL_AWsT8Sxhmy4dVsINHVhy6sxf_JVXZy_x3RM2k'}
Feb 27 2021 11:15:59.362138 [DBG 1] perf: Starting for frame:alarm
Feb 27 2021 11:15:59.362256 [DBG 1] ============ Frame: alarm Running object model in sequence ==================
Feb 27 2021 11:15:59.362325 [DBG 3] load_models (just init, actual load happens at first detect): ['object']
Feb 27 2021 11:15:59.363140 [DBG 2] Initializing model  type:object with options:{'tpu_max_processes': 1, 'tpu_max_lock_wait': 120, 'gpu_max_processes': 1, 'gpu_max_lock_wait': 120, 'cpu_max_processes': 3, 'cpu_max_lock_wait': 120, 'max_detection_size': '90%', 'object_config': 'models/yolov3/yolov3.cfg', 'object_weights': 'models/yolov3/yolov3.weights', 'object_labels': 'models/yolov3/coco.names', 'object_min_confidence': 0.4, 'object_framework': 'opencv', 'object_processor': 'cpu', 'disable_locks': 'no'}
Feb 27 2021 11:15:59.370338 [DBG 2] portalock: max:3, name:pyzm_uid1000_cpu_lock, timeout:120
Feb 27 2021 11:15:59.370421 [DBG 2] Waiting for pyzm_uid1000_cpu_lock portalock...
Feb 27 2021 11:15:59.370750 [DBG 2] Got pyzm_uid1000_cpu_lock portalock
Feb 27 2021 11:15:59.370808 [DBG 3] object has a same_model_sequence strategy of first
Feb 27 2021 11:15:59.370852 [DBG 3] --------- Frame:alarm Running variation: #1 -------------
Feb 27 2021 11:15:59.370909 [DBG 2] pyzm_uid1000_cpu_lock portalock already acquired
Feb 27 2021 11:15:59.370953 [DBG 1] |--------- Loading Yolo model from disk ---------|
Feb 27 2021 11:15:59.483256 [DBG 1] perf: processor:cpu Yolo initialization (loading models/yolov3/yolov3.weights model from disk) took: 112.14 ms
Feb 27 2021 11:15:59.483424 [DBG 1] Using CPU for detection
Feb 27 2021 11:15:59.483593 [DBG 1] |---------- YOLO (input image: 640w*480h, model resize dimensions: 416w*416h) ----------|
Feb 27 2021 11:16:00.375669 [DBG 2] Released pyzm_uid1000_cpu_lock portalock
Feb 27 2021 11:16:00.375733 [DBG 1] perf: processor:cpu Yolo detection took: 892.10 ms
Feb 27 2021 11:16:00.499933 [DBG 2] perf: processor:cpu Yolo NMS filtering took: 0.88 ms
Feb 27 2021 11:16:00.500911 [DBG 3] core model detection over, got 3 objects. Now filtering
Feb 27 2021 11:16:00.500945 [DBG 3] Max object size found to be: 90%
Feb 27 2021 11:16:00.501242 [DBG 2] Converted 90% to 276480.0
Feb 27 2021 11:16:00.501310 [DBG 2] Ignoring pottedplant [362, 392, 582, 478] as conf. level 0.22285059094429016 is lower than 0.4
Feb 27 2021 11:16:00.501329 [DBG 2] Returning filtered list of 2 objects.
Feb 27 2021 11:16:00.501354 [DBG 4] This model iteration inside object found: labels: ['car', 'person'],conf:[0.7734057903289795, 0.49510592222213745]
Feb 27 2021 11:16:00.501383 [DBG 2] No polygons, adding full image polygon: {'name': 'full_image', 'value': [(0, 0), (640, 0), (640, 480), (0, 480)], 'pattern': None}
Feb 27 2021 11:16:00.501597 [DBG 3] intersection: comparing object:car,POLYGON ((162 3, 364 3, 364 75, 162 75, 162 3)) to polygon:full_image,POLYGON ((0 0, 640 0, 640 480, 0 480, 0 0))
Feb 27 2021 11:16:00.501642 [DBG 2] Match pattern overridden to (person|bear|dog|cat|car|bicycle|motorbike|truck) in ml_overrides
Feb 27 2021 11:16:00.501853 [DBG 3] full_image intersects object:car[[(162, 3), (364, 3), (364, 75), (162, 75)]]
Feb 27 2021 11:16:00.502012 [DBG 3] intersection: comparing object:person,POLYGON ((7 312, 175 312, 175 462, 7 462, 7 312)) to polygon:full_image,POLYGON ((0 0, 640 0, 640 480, 0 480, 0 0))
Feb 27 2021 11:16:00.502040 [DBG 2] Match pattern overridden to (person|bear|dog|cat|car|bicycle|motorbike|truck) in ml_overrides
Feb 27 2021 11:16:00.502074 [DBG 3] full_image intersects object:person[[(7, 312), (175, 312), (175, 462), (7, 462)]]
Feb 27 2021 11:16:00.502093 [DBG 3] breaking out of same model loop, as matches found and strategy is "first"
Feb 27 2021 11:16:00.502259 [DBG 3] Reading http://192.168.1.189:8095/index.php?view=image&eid=622971&fid=snapshot
Feb 27 2021 11:16:00.502281 [DBG 3] No need to relogin as access token still has 119.86287376666668 minutes remaining
Feb 27 2021 11:16:00.502302 [DBG 4] make_request called with url=http://192.168.1.189:8095/index.php?view=image&eid=622971&fid=snapshot payload={} type=get query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjE0NDQ2MTUyLCJleHAiOjE2MTQ0NTMzNTIsInVzZXIiOiJhZG1pbiIsInR5cGUiOiJhY2Nlc3MifQ.N-POL_AWsT8Sxhmy4dVsINHVhy6sxf_JVXZy_x3RM2k'}
Feb 27 2021 11:16:00.525457 [DBG 1] perf: Starting for frame:snapshot
Feb 27 2021 11:16:00.525501 [DBG 1] ============ Frame: snapshot Running object model in sequence ==================
Feb 27 2021 11:16:00.525515 [DBG 3] object has a same_model_sequence strategy of first
Feb 27 2021 11:16:00.525524 [DBG 3] --------- Frame:snapshot Running variation: #1 -------------
Feb 27 2021 11:16:00.525539 [DBG 2] Waiting for pyzm_uid1000_cpu_lock portalock...
Feb 27 2021 11:16:00.525685 [DBG 2] Got pyzm_uid1000_cpu_lock portalock
Feb 27 2021 11:16:00.525703 [DBG 1] |---------- YOLO (input image: 640w*480h, model resize dimensions: 416w*416h) ----------|
Feb 27 2021 11:16:01.157419 [DBG 2] Released pyzm_uid1000_cpu_lock portalock
Feb 27 2021 11:16:01.157488 [DBG 1] perf: processor:cpu Yolo detection took: 631.76 ms
Feb 27 2021 11:16:01.295411 [DBG 2] perf: processor:cpu Yolo NMS filtering took: 0.76 ms
Feb 27 2021 11:16:01.296277 [DBG 3] core model detection over, got 3 objects. Now filtering
Feb 27 2021 11:16:01.296306 [DBG 3] Max object size found to be: 90%
Feb 27 2021 11:16:01.296376 [DBG 2] Converted 90% to 276480.0
Feb 27 2021 11:16:01.296404 [DBG 2] Ignoring pottedplant [370, 392, 574, 478] as conf. level 0.3035900890827179 is lower than 0.4
Feb 27 2021 11:16:01.296418 [DBG 2] Returning filtered list of 2 objects.
Feb 27 2021 11:16:01.296439 [DBG 4] This model iteration inside object found: labels: ['person', 'car'],conf:[0.9910268783569336, 0.8752051591873169]
Feb 27 2021 11:16:01.296651 [DBG 3] intersection: comparing object:person,POLYGON ((126 200, 230 200, 230 412, 126 412, 126 200)) to polygon:full_image,POLYGON ((0 0, 640 0, 640 480, 0 480, 0 0))
Feb 27 2021 11:16:01.296705 [DBG 2] Match pattern overridden to (person|bear|dog|cat|car|bicycle|motorbike|truck) in ml_overrides
Feb 27 2021 11:16:01.296743 [DBG 3] full_image intersects object:person[[(126, 200), (230, 200), (230, 412), (126, 412)]]
Feb 27 2021 11:16:01.296856 [DBG 3] intersection: comparing object:car,POLYGON ((162 3, 354 3, 354 75, 162 75, 162 3)) to polygon:full_image,POLYGON ((0 0, 640 0, 640 480, 0 480, 0 0))
Feb 27 2021 11:16:01.296904 [DBG 2] Match pattern overridden to (person|bear|dog|cat|car|bicycle|motorbike|truck) in ml_overrides
Feb 27 2021 11:16:01.296926 [DBG 3] full_image intersects object:car[[(162, 3), (354, 3), (354, 75), (162, 75)]]
Feb 27 2021 11:16:01.296948 [DBG 3] breaking out of same model loop, as matches found and strategy is "first"
Feb 27 2021 11:16:01.297130 [DBG 2] pyzm_uid1000_cpu_lock portalock already released
Feb 27 2021 11:16:01.297155 [DBG 1] perf: TOTAL detection sequence (with image loads) took: 1978.35 ms  to process 622971
Feb 27 2021 11:16:01.297457 [DBG 1] Returning {'matched_data': {'boxes': [[162, 3, 364, 75], [7, 312, 175, 462]], 'error_boxes': [], 'labels': ['car', 'person'], 'confidences': [0.7734057903289795, 0.49510592222213745], 'frame_id': 'alarm', 'image_dimensions': {'original': (480, 640), 'resized': (480, 640)}, 'image': None}, 'all_matches': [{'frame_id': 'alarm', 'boxes': [[162, 3, 364, 75], [7, 312, 175, 462]], 'error_boxes': [], 'labels': ['car', 'person'], 'confidences': [0.7734057903289795, 0.49510592222213745], 'models': ['object']}, {'frame_id': 'snapshot', 'boxes': [[126, 200, 230, 412], [162, 3, 354, 75]], 'error_boxes': [], 'labels': ['person', 'car'], 'confidences': [0.9910268783569336, 0.8752051591873169], 'models': ['object']}], 'polygons': []}

If I understand it right, it seems mlapi compares the detected polygon (eg: person) to to the full image, and does not take into account the defined zones : [DBG 2] No polygons, adding full image polygon: {'name': 'full_image', 'value': [(0, 0), (640, 0), (640, 480), (0, 480)], 'pattern': None} and intersection: comparing object:person,POLYGON ((126 200, 230 200, 230 412, 126 412, 126 200)) to polygon:full_image,POLYGON ((0 0, 640 0, 640 480, 0 480, 0 0))

Also, running the same detection with monitor defined in mlapiconfig.ini it crashes on :

Feb 27 2021 11:24:37.315847 [DBG 3] intersection: comparing object:car,POLYGON ((162 3, 364 3, 364 75, 162 75, 162 3)) to polygon:smaller_m10,POLYGON ((535 64, 597 57, 543 474, 362 476, 535 64))
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router
    return original_handler(e)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.9/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 272, in error_router
    return original_handler(e)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3.9/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 468, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper
    return fn(*args, **kwargs)
  File "/home/x/.mlapi/mlapi.py", line 229, in post
    matched_data,all_matches = m.detect_stream(stream=stream, options=stream_options, ml_overrides=ml_overrides)
  File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 466, in detect_stream
    _b,_l,_c, _e = self._filter_patterns(seq,_b,_l,_c, polygons)
  File "/usr/lib/python3.9/site-packages/pyzm/ml/detect_sequence.py", line 263, in _filter_patterns
    poly = Polygon(p['value'])
KeyError: 'value'
pliablepixels commented 3 years ago
  1. Are you sure you updated correct? I bumped up the version now to make sure. You should see 2.2.4
  2. Don't specify monitor sections in objectconfig, put it in mlapiconfig
  3. If pt 2 is resulting in a crash, please post debug logs of the run - not just the error .
mfauvain commented 3 years ago

Just reinstalled the latest 2.2.4 --------| mlapi version:2.2.4, pyzm version:0.3.35 |-------- and added the monitor section in mlapiconfig.ini. This time no crash, and it worls (ie : car/person detected does not intersect defined polygon)

Feb 27 2021 18:08:18.882109 [DBG 3] intersection: comparing object:person,POLYGON ((126 200, 230 200, 230 412, 126 412, 126 200)) to polygon:smaller_m10,POLYGON ((535 64, 597 57, 543 474, 362 476, 535 64))
Feb 27 2021 18:08:18.882214 [INF] object:person at POLYGON ((126 200, 230 200, 230 412, 126 412, 126 200)) does not fall into any polygons, removing...
Feb 27 2021 18:08:18.882339 [DBG 3] intersection: comparing object:car,POLYGON ((162 3, 354 3, 354 75, 162 75, 162 3)) to polygon:smaller_m10,POLYGON ((535 64, 597 57, 543 474, 362 476, 535 64))
Feb 27 2021 18:08:18.882388 [INF] object:car at POLYGON ((162 3, 354 3, 354 75, 162 75, 162 3)) does not fall into any polygons, removing...
Feb 27 2021 18:08:18.882419 [DBG 2] We did not find any object matches in frame: snapshot
Feb 27 2021 18:08:18.882438 [DBG 2] pyzm_uid1000_cpu_lock portalock already released
Feb 27 2021 18:08:18.882459 [DBG 1] perf: TOTAL detection sequence (with image loads) took: 2225.06 ms  to process 622971
Feb 27 2021 18:08:18.882487 [DBG 4] Restoring global config & ml_options
Feb 27 2021 18:08:18.882531 [DBG 1] Returning {'matched_data': {'boxes': [], 'error_boxes': [], 'labels': [], 'confidences': [], 'frame_id': None, 'image_dimensions': {'original': (480, 640), 'resized': (480, 640)}, 'image': None}, 'all_matches': [], 'polygons': [{'name': 'smaller_m10', 'value': [(535, 64), (597, 57), (543, 474), (362, 476)], 'pattern': '(person|bear|dog|cat|car|bicycle|motorbike|truck)'}]}

However, removing the monitor section from mlapiconfig.ini, mlapi uses full_image...

intersection: comparing object:person,POLYGON ((126 200, 230 200, 230 412, 126 412, 126 200)) to polygon:full_image,POLYGON ((0 0, 640 0, 640 480, 0 480, 0 0))
Feb 27 2021 18:12:52.931345 [DBG 2] Match pattern overridden to (person|bear|dog|cat|car|bicycle|motorbike|truck) in ml_overrides
Feb 27 2021 18:12:52.931376 [DBG 3] full_image intersects object:person[[(126, 200), (230, 200), (230, 412), (126, 412)]]
Feb 27 2021 18:12:52.931478 [DBG 3] intersection: comparing object:car,POLYGON ((162 3, 354 3, 354 75, 162 75, 162 3)) to polygon:full_image,POLYGON ((0 0, 640 0, 640 480, 0 480, 0 0))
Feb 27 2021 18:12:52.931503 [DBG 2] Match pattern overridden to (person|bear|dog|cat|car|bicycle|motorbike|truck) in ml_overrides
Feb 27 2021 18:12:52.931521 [DBG 3] full_image intersects object:car[[(162, 3), (354, 3), (354, 75), (162, 75)]]
Feb 27 2021 18:12:52.931540 [DBG 3] breaking out of same model loop, as matches found and strategy is "first"
Feb 27 2021 18:12:52.931704 [DBG 2] pyzm_uid1000_cpu_lock portalock already released
Feb 27 2021 18:12:52.931725 [DBG 1] perf: TOTAL detection sequence (with image loads) took: 2136.84 ms  to process 622971
Feb 27 2021 18:12:52.931986 [DBG 1] Returning {'matched_data': {'boxes': [[162, 3, 364, 75], [7, 312, 175, 462]], 'error_boxes': [], 'labels': ['car', 'person'], 'confidences': [0.7734057903289795, 0.49510592222213745], 'frame_id': 'alarm', 'image_dimensions': {'original': (480, 640), 'resized': (480, 640)}, 'image': None}, 'all_matches': [{'frame_id': 'alarm', 'boxes': [[162, 3, 364, 75], [7, 312, 175, 462]], 'error_boxes': [], 'labels': ['car', 'person'], 'confidences': [0.7734057903289795, 0.49510592222213745], 'models': ['object']}, {'frame_id': 'snapshot', 'boxes': [[126, 200, 230, 412], [162, 3, 354, 75]], 'error_boxes': [], 'labels': ['person', 'car'], 'confidences': [0.9910268783569336, 0.8752051591873169], 'models': ['object']}], 'polygons': []}

As I use the 'ml_fallback' option it is better for me to have the monitors defined in objectconfig.ini and not on the mlapi server. Will use this in the meantime. Let me know if you need full logs

pliablepixels commented 3 years ago

Correct. Always put it in mlapiconfig. If you want fallback put it in both. I’m planning to discontinue support of mlapi picking from objectconfig. It gets too confusing.

mfauvain commented 3 years ago

Ok, leaving it there then. It's not like I modify the monitors specs often anyway, so good for me. Thx for the help.