kestra-io / kestra

Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code and from the UI.
https://kestra.io
Apache License 2.0
7.68k stars 470 forks source link

Simple Execution running forever, nothing in logs #3664

Closed andeke07 closed 4 months ago

andeke07 commented 4 months ago

Describe the issue

I have just started playing around with Kestra and have gotten it running in my k3s cluster using the Helm chart. The only settings I changed were to add my ingress and separate out the services rather than running all-in-one.

I followed the getting started tutorial after everything launched but found even the demonstration executions were hanging on the first steps. No logs, nothing, just running forever.

Even a simple execution:

id: myflow
namespace: myteam
description: Save and Execute the flow

labels:
  env: dev
  project: myproject

tasks:
  - id: print_status
    type: io.kestra.core.tasks.log.Log
    message: hellohihihi

Is not logging anything (notice is has been running for 27 minutes now:

image

I tried to increase the logging level in the Helm chart:

configuration:
  logger:
    levels:
      io.kestra.runner: TRACE

But I am not seeing anything extra in my executor pod. The last thing I see in the executor pod:

2024-05-04 10:04:30,659 INFO  jdbc-queue-Execution_0 flow.myflow  [namespace: myteam] [flow: myflow] [execution: 3p1tByRVE0J1GtX7F6GmQQ] Flow started

Am I missing something here? Here is my full helm chart:

affinity: {}
annotations: {}
configuration:
  logger:
    levels:
      io.kestra.runner: TRACE
configurationPath: null
deployments:
  executor:
    command: server executor
    enabled: true
    kind: Deployment
  indexer:
    command: server indexer
    enabled: true
    kind: Deployment
  scheduler:
    command: server scheduler
    enabled: true
    kind: Deployment
  standalone:
    command: >-
      server standalone --worker-thread={{
      .Values.deployments.standalone.workerThreads }}
    enabled: false
    kind: Deployment
    terminationGracePeriodSeconds: 60
    workerThreads: 128
  webserver:
    affinity: {}
    autoscaler:
      enabled: false
      extra: {}
      maxReplicas: 3
      metrics: []
      minReplicas: 1
    command: server webserver
    enabled: true
    extraContainers: []
    extraEnv: []
    kind: Deployment
    nodeSelector: {}
    podSecurityContext: {}
    replicaCount: 1
    resources: {}
    securityContext: {}
    strategy: {}
    terminationGracePeriodSeconds: 30
    tolerations: []
  worker:
    command: server worker --thread={{ .Values.deployments.worker.workerThreads }}
    enabled: false
    kind: Deployment
    terminationGracePeriodSeconds: 60
    workerThreads: 128
dind:
  args:
    - '--log-level=fatal'
    - '--group=1000'
  enabled: true
  extraEnv: []
  extraVolumeMounts: []
  image:
    image: docker
    pullPolicy: IfNotPresent
    tag: dind-rootless
  resources: {}
  securityContext:
    runAsGroup: 1000
    runAsUser: 1000
  socketPath: /dind/
  tmpPath: /tmp/
elasticsearch:
  antiAffinity: soft
  clusterName: es-kestra
  enabled: false
  esConfig:
    elasticsearch.yml: |
      xpack:
        security:
          enabled: false
  roles:
    ml: 'false'
executable: /app/kestra
externalSecret: {}
extraConfigMapEnvFrom: null
extraContainers: []
extraEnv: []
extraSecretEnvFrom: null
extraVolumeMounts: []
extraVolumes: []
image:
  image: kestra/kestra
  pullPolicy: Always
  tag: latest-full
imagePullSecrets: []
ingress:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
  className: ''
  enabled: true
  hosts:
    - host: my.internal.url
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls:
    - hosts:
        - my.internal.url
      secretName: kestra-ingress-tls
initContainers: []
kafka:
  enabled: false
livenessProbe:
  enabled: true
  failureThreshold: 3
  httpGetExtra: {}
  initialDelaySeconds: 0
  path: /health
  periodSeconds: 5
  port: management
  successThreshold: 1
  timeoutSeconds: 3
minio:
  buckets:
    - name: kestra
      policy: none
      purge: false
  enabled: true
  mode: standalone
  persistence:
    size: 8Gi
  resources:
    requests:
      memory: 512Mi
  rootPassword: its-not-a-secret
  rootUser: please-change-me
nameOverride: ''
nodeSelector: {}
podAnnotations: {}
podSecurityContext: {}
postgresql:
  auth:
    database: kestra
    password: kestra
    username: kestra
  enabled: true
  primary:
    persistence:
      enabled: true
      size: 8Gi
readinessProbe:
  enabled: true
  failureThreshold: 3
  httpGetExtra: {}
  initialDelaySeconds: 0
  path: /health
  periodSeconds: 5
  port: management
  successThreshold: 1
  timeoutSeconds: 3
resources: {}
secrets: {}
securityContext:
  runAsGroup: 1000
  runAsUser: 1000
service:
  annotations: {}
  loadBalancerIP: ''
  port: 8080
  type: ClusterIP
serviceAccountName: ''
startupProbe:
  enabled: true
  failureThreshold: 120
  httpGetExtra: {}
  initialDelaySeconds: 1
  path: /health
  periodSeconds: 1
  port: management
  successThreshold: 1
  timeoutSeconds: 1
terminationGracePeriodSeconds: 30
tolerations: []
workerGroup:
  workergroupname:
    command: >-
      server worker --thread={{
      .Values.workerGroup.workergroupname.workerThreads }} --worker-group={{
      .WorkerGroup }}
    enabled: false
    kind: Deployment
    terminationGracePeriodSeconds: 60
    workerThreads: 128
global:
  cattle:
    systemProjectId: p-bmmdg

Environment

andeke07 commented 4 months ago

Oops, having read through my Helm chart again I have only just now seen that I didn't enable the worker service...