search

OpenVisualCloud / Smart-City-Sample

The smart city reference pipeline shows how to integrate various media building blocks, with analytics powered by the OpenVINO™ Toolkit, for traffic or stadium sensing, analytics and management tasks.
BSD 3-Clause "New" or "Revised" License
185 stars 81 forks source link

Error while loading Pipeline - YoloV3 model #782

Closed KailashGanapathy closed 2 years ago

KailashGanapathy commented 2 years ago

Hello all,

I tried to load YoloV3 (converted IR Model), in CPU, Model Network: FP32

Open_model_zoo: https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolo-v3-tf Tensorflow model: https://download.01.org/opencv/public_models/022020/yolo_v3/yolov3.pb Json file: https://download.01.org/opencv/public_models/022020/yolo_v3/yolo_v3_new.json

IR_Conversion_Script: python3 /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo_tf.py --input_shape [1,416,416,3] --input input_1 --scale_values input_1[255] --reverse_input_channels --transformations_config ./OMZ_YOLOV3_TF_Model/yolo_v3_new.json --input_model ./OMZ_YOLOV3_TF_Model/yolov3.pb --output_dir ./IR/

I tried removing INT8 model and forcely kept FP32 model,

Error log from OVC:

Generating LALR tables Searching... [INFO] sensor msg: rtsp Connected to BF9S4XoBjrHmACLjMiHW... testing mqtt connection mqtt connected {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,599", "message": "========================", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,599", "message": "Options for vaserving.py", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,599", "message": "========================", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,599", "message": "port == 8080", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,599", "message": "framework == gstreamer", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,600", "message": "pipeline_dir == /home/pipelines", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,600", "message": "model_dir == /home/models", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,600", "message": "network_preference == {'CPU': 'INT8,FP32'}", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,600", "message": "max_running_pipelines == 1", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,600", "message": "log_level == INFO", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,600", "message": "config_path == /home/vaserving/..", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,600", "message": "ignore_init_errors == False", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,600", "message": "========================", "module": "vaserving"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,601", "message": "==============", "module": "model_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,601", "message": "Loading Models", "module": "model_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,601", "message": "==============", "module": "model_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,601", "message": "Loading Models from Path /home/models", "module": "model_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,653", "message": "Loading Model: object_detection_2020R2 version: 1 type: IntelDLDT from {'FP32': '/home/models/object_detection_2020R2/1/FP32/yolov3.xml', 'model-proc': '/home/models/object_detection_2020R2/1/yolov3.json'}", "module": "model_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,654", "message": "========================", "module": "model_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,654", "message": "Completed Loading Models", "module": "model_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,654", "message": "========================", "module": "model_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,655", "message": "=================", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,655", "message": "Loading Pipelines", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:14,655", "message": "=================", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:19,135", "message": "Loading Pipelines from Config Path /home/pipelines", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:20,823", "message": "Loading Pipeline: object_detection version: 4 type: GStreamer from /home/pipelines/object_detection/4/pipeline.json", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:21,108", "message": "Loading Pipeline: object_detection version: 2 type: GStreamer from /home/pipelines/object_detection/2/pipeline.json", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:21,207", "message": "Loading Pipeline: object_detection version: 1 type: GStreamer from /home/pipelines/object_detection/1/pipeline.json", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:21,263", "message": "Loading Pipeline: object_detection version: 3 type: GStreamer from /home/pipelines/object_detection/3/pipeline.json", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:21,263", "message": "===========================", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:21,263", "message": "Completed Loading Pipelines", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:21,264", "message": "===========================", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:21,264", "message": "Creating Instance of Pipeline object_detection/1", "module": "pipeline_manager"} {"levelname": "INFO", "asctime": "2021-07-26 11:10:21,296", "message": "Device preferred network INT8 not found", "module": "model_manager"} PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=6.866455078125e-05, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) on_created: /tmp/rec/BF9S4XoBjrHmACLjMiHW/2021/07/26/1627278024244428192_1474343959.mp4 PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=3.0013587474823, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=6.001809597015381, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=9.002149820327759, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=12.002528190612793, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=15.002888441085815, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=18.003196477890015, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=21.003546237945557, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=24.0038800239563, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=27.004395008087158, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=30.004823684692383, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=33.005263805389404, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=36.00559902191162, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=39.00599455833435, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=42.00645327568054, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=45.006819009780884, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=48.007275342941284, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=51.007657051086426, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=54.00809383392334, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=57.00853681564331, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=60.008960485458374, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=63.009300231933594, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=66.00956678390503, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=69.01006317138672, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=72.01044178009033, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=75.01072978973389, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=78.01087641716003, id=1, start_time=1627278021.299261, state=<State.QUEUED: 1>) {"levelname": "ERROR", "asctime": "2021-07-26 11:11:40,513", "message": "Error on Pipeline 1: gst-library-error-quark: base_inference plugin intitialization failed (3): /opt/build/gst-video-analytics/gst/inference_elements/base/inference_singleton.cpp(137): acquire_inference_instance (): /GstPipeline:pipeline4/GstGvaDetect:detection:\nFailed to load model '/home/models/object_detection_2020R2/1/FP32/yolov3.xml'\n\tCannot create Gather layer up_sampling2d/Shape/GatherNCHWtoNHWC id:400 from unsupported opset: opset7\n", "module": "gstreamer_pipeline"} PipelineStatus(avg_fps=0, avg_pipeline_latency=None, elapsed_time=79.21747660636902, id=1, start_time=1627278021.299261, state=<State.ERROR: 4>) Pipeline object_detection Version 1 Instance 1 Ended with ERROR Traceback (most recent call last): File "/home/detect-object.py", line 32, in connect raise Exception("VA exited. This should not happen.") Exception: VA exited. This should not happen.

Model-proc Json file used:

{ "json_schema_version": "2.0.0", "input_preproc": [], "output_postproc": [ { "converter": "RegionYolo", "iou_threshold": 0.5, "classes": 80, "anchors": [ 10.0, 13.0, 16.0, 30.0, 33.0, 23.0, 30.0, 61.0, 62.0, 45.0, 59.0, 119.0, 116.0, 90.0, 156.0, 198.0, 373.0, 326.0 ], "masks": [ 6, 7, 8, 3, 4, 5, 0, 1, 2 ], "bbox_number_on_cell": 3, "cells_number": 13, "labels": [ "person", "bicycle", "car", "motorbike", "aeroplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "sofa", "pottedplant", "bed", "diningtable", "toilet", "tvmonitor", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush" ] } ] }

It would be much appreciated if you could tell me where am I going wrong,

KailashGanapathy commented 2 years ago

Hello all,

I recently came to know OVC supports OpenVINO 2021.2.

The sample models available in OVC are mostly SSD based object detections, could anyone please confirm that OVC supports Yolo models too,

I tried loading person-vehicle-bike-detection-crossroad-yolov3-1020 and model-proc but couldn't achieve it.

FYI YoloV3 model loading discussion.

Does the Model-proc file really need for gvadetect, which holds layer names, converter, and labels in it?

KailashGanapathy commented 2 years ago

Hello all,

Just to give an update, I'm able to load yolov3 models in OVC, and execute it.

By changing the entry points in yolov3.json. I used yolo-v3-tf, have generated IR for openvino 2021.2 and model-proc

I hope, this will help you !