[Bug] OVMS operator doesn't support `batch_size` and `layout` options

openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

https://docs.openvino.ai

Apache License 2.0

6.54k stars 2.12k forks source link

[Bug] OVMS operator doesn't support `batch_size` and `layout` options #8607

Closed stefannica closed 2 years ago

stefannica commented 2 years ago

System information (version)

OVMS Operator image => 0.1.0

Detailed description

The latest version of OVMS operator (0.1.0) does not support the batch_size and layout server options.

Steps to reproduce

Install the latest version of the OVMS operator (0.1.0)
Configure an Ovms resource and specify custom values for the batch_size and layout attributes (e.g. batch_size: auto and layout: NHWC)
The ovms command line arguments generated in the OVMS deployment/pod do not reflect the batch_size nor the layout values used as input

Issue submission checklist

[x] I report the issue, it's not a question
[x] I checked the problem with documentation, FAQ, open issues, Stack Overflow, etc and have not found solution
[ ] There is reproducer code and related data files: images, videos, models, etc.

jgespino commented 2 years ago

Hi @stefannica

I see the latest ovms operator is 0.2.0, do you see the same behavior the latest version? https://catalog.redhat.com/software/containers/intel/ovms-operator/60793dc3ecb524169918956a?container-tabs=technical-information

Regards, Jesus

stefannica commented 2 years ago

Hi @jgespino , thanks for the quick reply.

I'm using the public container image published at https://quay.io/repository/openvino/ovms-operator?tab=tags. I don't see a 0.2.0 version there yet. Is there another official container image registry/source I can use ?

dtrawins commented 2 years ago

@stefannica The OpenVINO toolkit operator got upgraded to version 0.2.0 on OpenShift platform offering https://catalog.redhat.com/software/operators/detail/60649a56209af65d24b7ca9e In that version it does include parameters batch_size and layout. If you have an openshift cluster, you can install it from the list of certified operators.

The operator for opensource kubernetes is planned to be upgraded soon. Probably in few weeks. Till that time you might consider using a helm chart to deploy the model servers from https://github.com/openvinotoolkit/model_server/tree/main/deploy.

stefannica commented 2 years ago

Thanks @dtrawins , I will wait for the opensource kubernetes version to be upgraded. I'm using OVMS as one of several opensource tools integrated with the FuseML MLOps orchestration framework: https://fuseml.github.io/docs/v0.3/tutorials/openvino-extensions/ . I'll update the FuseML extension when 0.2.0 is released.

dtrawins commented 2 years ago

@stefannica The openvino operator on opertaorhub is now updated and includes batch_size and layout parameters