search

openvinotoolkit / model_server

A scalable inference server for models optimized with OpenVINO™
https://docs.openvino.ai/2024/ovms_what_is_openvino_model_server.html
Apache License 2.0
687 stars 212 forks source link

Failed to find a working yolov8 client on model server. #2195

Open charlestamz opened 1 year ago

charlestamz commented 1 year ago

I generated a yolov8 int8 OpenVINO model based on the following link, but the triton server crashed when deploying it. https://docs.openvino.ai/2023.2/notebooks/122-yolov8-quantization-with-accuracy-control-with-output.html

and I tried to deploy it on model server and the server looks ok. then I tried many clients in the demo. but all failed. Can anyone give some hints or a working example?

yolov8n.zip

jiekechoo commented 1 month ago

I have the same issue. OVMS: v2024.3

[2024-10-17 10:30:34.491][102][modelmanager][error][modelinstance.cpp:368] Error during layout configuration
[2024-10-17 10:30:34.491][102][serving][info][modelversionstatus.cpp:109] STATUS CHANGE: Version 1 of model yolo11n-int8 status change. New status: ( "state": "LOADING", "error_code": "UNKNOWN" )
[2024-10-17 10:30:34.491][102][serving][error][model.cpp:276] Error occurred while loading model: yolo11n-int8; version: 1; error: Unknown error
[2024-10-17 10:30:34.491][102][modelmanager][error][modelmanager.cpp:1365] Error occurred while reloading model: yolo11n-int8; versions; error: Unknown error
[2024-10-17 10:30:34.491][102][serving][info][model.cpp:190] Will clean up model: yolo11n-int8; version: 1 ...
[2024-10-17 10:30:34.491][102][serving][info][model.cpp:88] Updating default version for model: yolo11n-int8, from: 0
[2024-10-17 10:30:34.491][102][serving][info][model.cpp:100] Model: yolo11n-int8 will not have default version since no version is available.
[2024-10-17 10:30:34.491][102][serving][info][modelversionstatus.cpp:109] STATUS CHANGE: Version 1 of model yolo11n-int8 status change. New status: ( "state": "LOADING", "error_code": "UNKNOWN" )
mzegla commented 1 month ago

As far as I know YoloV11 is not supported by OV now. Version 2024.5 will likely support it.

jiekechoo commented 1 month ago

Which version of yolo is supported by current ovms?

mzegla commented 1 month ago

Check out: https://docs.openvino.ai/2024/about-openvino/compatibility-and-support/supported-models.html You can search for the model of interest on that list.