nekomeowww / ollama-operator

Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫
https://ollama-operator.ayaka.io
Apache License 2.0
113 stars 13 forks source link

Cannot deploy model that contains dots/colons in its name #21

Open alejandro-lopez-intellecteu opened 7 hours ago

alejandro-lopez-intellecteu commented 7 hours ago

Hi, I'm trying to deploy some models using the operator, but I'm facing some naming issues if these models contain dots or invalid caracters for kubernetes spec.metadata.name. Some examples:

Example to deploy qwen2.5: https://ollama.com/library/qwen2.5

Model CRD:

apiVersion: ollama.ayaka.io/v1
kind: Model
metadata:
  name: 'qwen2.5'
spec:
  image: qwen2.5
  persistentVolume:
    accessMode: ReadWriteOnce

I receive:

2024-11-19T14:31:45Z ERROR Reconciler error {"controller": "model", "controllerGroup": "ollama.ayaka.io", "controllerKind": "Model", "Model": {"name":"qwen2.5","namespace":"default"}, "namespace": "default", "name": "qwen2.5", "reconcileID": "bdd6e0f4-05b3-49e0-ae01-e3db0aaae0ab", "error": "Service \"ollama-model-qwen2.5\" is invalid: metadata.name: Invalid value: \"ollama-model-qwen2.5\": a DNS-1035 label must consist of lower case alphanumeric characters or '-', start with an alphabetic character, and end with an alphanumeric character (e.g. 'my-name', or 'abc-123', regex used for validation is '[a-z]([-a-z0-9]*[a-z0-9])?')"}

So its k8s service is not created.

I'm also wondering how can I deploy a model that contains a colon, something like: qwen2.5:32b. It fails due to the same reason, metadata.name can't contain colons.

I see that we have this init command harcoded in the operator: fmt.Sprintf("apt update && apt install curl -y && ollama pull %s && curl http://ollama-models-store:11434/api/generate -d '{\"model\": \"%s\"}'"+image, name) and it fetches this name from the metadata.name field.

Can you help me?

Thanks!

nekomeowww commented 6 hours ago

Ah yes, you will have to alter the name to qwen2-5 or qwen25 to prevent this. Such issue was constraint by Kubernetes for the name of the resources.

For the metadata.name to preload the model, this is a bug and it should not happen. Let me fix it.