PrefectHQ / prefect-helm

Helm charts for deploying Prefect Services
Apache License 2.0
93 stars 58 forks source link

Helm Chart version 2024.4.25194423 references non existent docker image #323

Closed MrKriss closed 5 months ago

MrKriss commented 5 months ago

Looks like the latest helm charts try to install the bellow image for a prefect worker:

prefecthq/prefect:prefect-sqlalchemy-0.4.1-python3.11-kubernetes

However, at current, this image does not exist on docker hub and so any installs with this chart version are currently failing.

Looks like there might be some unintended consequences of how github builds get or set the Chart version and or the Prefect Version.

The fully rendered yaml config is shown below for creating a prefect worker:

---
# Source: prefect-worker/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: prefect-worker
  namespace: "default"
  labels:
    app.kubernetes.io/name: prefect-worker
    helm.sh/chart: prefect-worker-2024.4.25194423
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: "prefect-sqlalchemy-0.4.1"
    app.kubernetes.io/component: worker
    prefect-version: prefect-sqlalchemy-0.4.1
---
# Source: prefect-worker/templates/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: prefect-worker
  namespace: "default"
  labels:
    app.kubernetes.io/name: prefect-worker
    helm.sh/chart: prefect-worker-2024.4.25194423
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: "prefect-sqlalchemy-0.4.1"
    app.kubernetes.io/component: worker
    prefect-version: prefect-sqlalchemy-0.4.1
rules:
- apiGroups: [""]
  resources: ["events", "pods", "pods/log", "pods/status"]
  verbs: ["get", "watch", "list"]
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
# Source: prefect-worker/templates/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: prefect-worker
  namespace: "default"
  labels:
    app.kubernetes.io/name: prefect-worker
    helm.sh/chart: prefect-worker-2024.4.25194423
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: "prefect-sqlalchemy-0.4.1"
    app.kubernetes.io/component: worker
    prefect-version: prefect-sqlalchemy-0.4.1
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: prefect-worker
subjects:
  - kind: ServiceAccount
    name: prefect-worker
    namespace: "default"
---
# Source: prefect-worker/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prefect-worker
  namespace: "default"
  labels:
    app.kubernetes.io/name: prefect-worker
    helm.sh/chart: prefect-worker-2024.4.25194423
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: "prefect-sqlalchemy-0.4.1"
    app.kubernetes.io/component: worker
    prefect-version: prefect-sqlalchemy-0.4.1
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: prefect-worker
      app.kubernetes.io/instance: release-name
      app.kubernetes.io/component: worker
  template:
    metadata:
      labels:
        app.kubernetes.io/name: prefect-worker
        helm.sh/chart: prefect-worker-2024.4.25194423
        app.kubernetes.io/instance: release-name
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/version: "prefect-sqlalchemy-0.4.1"
        app.kubernetes.io/component: worker
        prefect-version: prefect-sqlalchemy-0.4.1
    spec:
      serviceAccountName: prefect-worker
      securityContext:
        fsGroup: 1001
        runAsNonRoot: true
        runAsUser: 1001
      containers:
        - name: prefect-worker
          image: "prefecthq/prefect:prefect-sqlalchemy-0.4.1-python3.11-kubernetes"
          imagePullPolicy: IfNotPresent
          command:
            - /usr/bin/tini
            - -g
            - --
            - /opt/prefect/entrypoint.sh
          args:
            - prefect
            - worker
            - start
            - --type
            - "kubernetes"
            - --pool
            - "my-workpool"
            - --install-policy
            - "prompt"
          workingDir: /home/prefect
          env:
            - name: HOME
              value: /home/prefect
            - name: PREFECT_WORKER_PREFETCH_SECONDS
              value: "10"
            - name: PREFECT_WORKER_QUERY_SECONDS
              value: "5"
            - name: PREFECT_API_ENABLE_HTTP2
              value: "true"
            - name: PREFECT_API_URL
              value: "http://prefect-server.default.svc.cluster.local:4200/api"
            - name: PREFECT_KUBERNETES_CLUSTER_UID
              value: ""
            - name: PREFECT_DEBUG_MODE
              value: "false"
          envFrom:
          resources:
            limits:
              cpu: 1000m
              memory: 1Gi
            requests:
              cpu: 100m
              memory: 256Mi
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 1001
          volumeMounts:
            - mountPath: /home/prefect
              name: scratch
              subPathExpr: home
            - mountPath: /tmp
              name: scratch
              subPathExpr: tmp
      volumes:
        - name: scratch
          emptyDir: {}
parkedwards commented 5 months ago

thanks for flagging @MrKriss - taking a look at this now

parkedwards commented 5 months ago

this was a result of our upstream repo changing their tag schema, and this has been fixed in #321

i've re-issued the worker chart version, so the new latest tag (2024.4.30151230) should have the correct app bundled

helm template eddie-worker prefect-oss/prefect-worker --version 2024.4.30151230 --set worker.config.workPool=default --set worker.cloudApiConfig.accountId=a-cool-id --set worker.cloudApiConfig.workspaceId=a-cool-id | grep 'image:'
          image: "prefecthq/prefect:2.18.1-python3.11-kubernetes"