Openshift and Kubernetes operator

The Operator installs and manages OpenVINO model servers in an OpenShift cluster and upstream Kubernetes. It enables inference execution at scale and exposes AI models via gRPC and REST API interfaces. The operator is using a model server helm chart which can be also used directly.

The Operator also integrates with the JupyterHub Spawner in Red Hat OpenShift Data Science and Open Data Hub. See detailed instructions below.

Operator installation

Installing and managing OpenVINO model server via the operator

Model server parameters

Autoscalability

Performance tuning

OpenVINO notebook integration with RHODS

Notebook parameters

Helm chart

Contact

If you have a question, a feature request, or a bug report, feel free to submit a Github issue.

• Other names and brands may be claimed as the property of others.