Ideally, we would like to follow what upstream KF does. This would eliminate the work we 'll have to do on each update as we would not have to maintain our own manifests but rather update only the controller's version (plus some authorization manifests).
Issue
Unfortunately, this is not possible in our case. Seldon controller creates resources (webhook, webhook-service) and expects find the specific hardcoded labels which is not the case for us. It also expects to find a deployment with a specific name, thus we would get a failed to find my deployment error. We do not use a deployment, instead we have the charm creating the container and the service pointing to its pod.
What we 'll do
Thus, we will diverge from upstream and keep applying resources manually, while also setting this env variable explicitly to false. During updates, what we can do is:
In a clean cluster, apply upstream KF manifests using kustomize build seldon-core-operator/base | kubectl apply -n kubeflow -f - from the kubeflow/manifests/contrib/seldon directory.
Check what resources have been created in the cluster (we can verify those by looking in the initializer function)
Go ahead and update our manifests accordingly
Note that:
we will still need to do a diff between old and new version KF manifests (as described in the Release handbook) for other manifests apart from those resources, like auth_manifests.
as with all sidecar charms, we do not include the deployment resource in our manifests since Pebble is responsible for creating the workload container in our case. However, we should consult any changes in the deployment that affect the container (e.g. its environment or command arguments).
Context
Since release 1.7, upstream KF has changed the way they deploy seldon-core component. Essentially, they stopped applying resouces manually(configmap, crds, webhook etc) and instead used the create_resources args in the container's exec command. What this means is that the controller will go ahead and initialize resources by itself using this initiliazer function.
Ideally, we would like to follow what upstream KF does. This would eliminate the work we 'll have to do on each update as we would not have to maintain our own manifests but rather update only the controller's version (plus some authorization manifests).
Issue
Unfortunately, this is not possible in our case. Seldon controller creates resources (webhook, webhook-service) and expects find the specific hardcoded labels which is not the case for us. It also expects to find a deployment with a specific name, thus we would get a failed to find my deployment error. We do not use a deployment, instead we have the charm creating the container and the service pointing to its pod.
What we 'll do
Thus, we will diverge from upstream and keep applying resources manually, while also setting this env variable explicitly to
false. During updates, what we can do is:kustomize build seldon-core-operator/base | kubectl apply -n kubeflow -f -from the kubeflow/manifests/contrib/seldon directory.Note that:
diffbetween old and new version KF manifests (as described in the Release handbook) for other manifests apart from those resources, likeauth_manifests.deploymentresource in our manifests since Pebble is responsible for creating the workload container in our case. However, we should consult any changes in the deployment that affect the container (e.g. its environment or command arguments).kustomize building manifests from seldon-core/operator/config/default