Creating Conductor Deployments and Services For Kubernetes

[mysteva]

I am currently evaluating Conductor as an orchestration framework, and am attempting to get all four containers (elasticsearch, dynomite, conductor-ui and conductor-server) to spin up successfully in Kubernetes. Is there a set of yaml files that exist that would provide the same functionality as the docker-compose.yaml file?

I've tried creating my own yaml files to apply to kubernetes, and they all seem to be running, but I can't get them communicating with each other (for instance, the ui clearly shows that there's no underlying data source). Any help would be greatly appreciated. Thanks!

[james-deee]

It sounds like it's not the yaml files you need help with then? It sounds like you don't have the correct configuration settings in your config.properties that got built into your conductor server image?

[mysteva]

@demichej It's possible. Maybe a better way to frame the question: is there a best practice that should be used for getting conductor, in its entirety, up and running in the cloud (whether it be AWS or Kubernetes)? Am I going about it the wrong way, and docker-compose is really the correct way to do it?

[james-deee]

I am not aware of a best practice @mysteva. We have decided to use Kubernetes to deploy our Conductor services + our workers.

We created our own yaml files for the Conductor services and we build it up with docker-compose and deploy manually.

I'm not sure if others would find it useful having some kind of "template" or something for their yaml files. Seems like a reasonable idea, but I dont think anything like that exists right now.

[mysteva]

@demichej Thanks for your response. Going forward, I do think that would be very valuable to include. Is there a way to make that request officially?

[james-deee]

You could take a stab at putting a PR together :)

I'm not sure of the best way to label this as an enhancement though. That's another option. I dont think i have the permissions to do that though.

[mysteva]

Ok, thanks. I don't suppose you'd by chance have your generic yaml files that you used, would you?

[apanicker-nflx]

@demichej Thanks for the suggestion. I have labeled this discussion as an enhancement. Please consider sharing your yaml files and Kubernetes deployment guide to benefit the community. This would be a great addition to the project.

[mysteva]

Hi, just wanted to check in to see if there were any sort of "stock" yaml files that we can use for testing via Kubernetes. Thanks!

[james-deee]

Hi @mysteva I'm still here :) I'm sorry I haven't had time to do that, but I will try and get this in a PR this week. Note though, we're using straight Redis (no dynomite), so I'm not sure if that will 100% solve your use case or not.

[eins]

Hello folks, any news about this ticket, conductor and kubernetes support ?

[apanicker-nflx]

@demichej Checking in to see if you would be able to pick this back up soon? Thanks.

[james-deee]

@apanicker-nflx Yeah it is on our list still, but has been pushed further down :(

Also, do y'all have plans to push a Docker image (maybe to the Netflix OSS Hub)? We are able to override all configuration from environment variables, so if an image was made available in the public domain, it would be pretty nice/easy to use.

Thanks.

[apanicker-nflx]

Thanks for the update, we are currently chalking out plans for Conductor 3.0 and publishing docker images as part of new releases is one of the items on the roadmap.

[mark91m12]

Hello guys, i'm doing exactly the same, it would be grate if one of you sharing your yaml files and Kubernetes deployment guide, any news about this? thank you in advance

[jasondavindev]

Elasticsearch

apiVersion: apps/v1
kind: Deployment
metadata:
  name: elasticsearch
  namespace: conductor
  labels:
    app: elasticsearch
spec:
  selector:
    matchLabels:
      app: elasticsearch
  replicas: 1
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
        - name: elasticsearch
          image: elasticsearch:6.8.15
          env:
            - name: "ES_JAVA_OPTS"
              value: "-Xms512m -Xmx1024m"
            - name: transport.host
              value: "0.0.0.0"
            - name: discovery.type
              value: "single-node"
            - name: xpack.security.enabled
              value: "false"
          ports:
            - containerPort: 9200
              name: http
            - containerPort: 9300
              name: custom
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  namespace: conductor
spec:
  selector:
    app: elasticsearch
  type: ClusterIP
  ports:
    - name: http
      protocol: TCP
      port: 9200

Postgres

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: conductor-db
  namespace: conductor
spec:
  selector:
    matchLabels:
      app: conductor-db
  serviceName: "conductor-db"
  replicas: 1
  template:
    metadata:
      labels:
        app: conductor-db
    spec:
      containers:
        - name: conductor-db
          image: postgres:11
          ports:
            - containerPort: 5432
              name: db
          env:
            - name: POSTGRES_USER
              value: conductor
            - name: POSTGRES_PASSWORD
              value: conductor
            - name: POSTGRES_DB
              value: conductor
            - name: PGDATA
              value: /data/pgdata
          volumeMounts:
            - name: data
              mountPath: /data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: "gp2"
        resources:
          requests:
            storage: 5Gi
---
apiVersion: v1
kind: Service
metadata:
  name: conductor-db
  namespace: conductor
spec:
  selector:
    app: conductor-db
  type: ClusterIP
  ports:
    - name: conductor-db
      protocol: TCP
      port: 5432

Conductor server

apiVersion: apps/v1
kind: Deployment
metadata:
  name: conductor-server
  namespace: conductor
  labels:
    app: conductor-server
spec:
  selector:
    matchLabels:
      app: conductor-server
  replicas: 1
  template:
    metadata:
      labels:
        app: conductor-server
    spec:
      containers:
        - name: conductor-server
          image: conductor-server
          resources:
            requests:
              cpu: 2
              memory: 4Gi
            limits:
              cpu: 2
              memory: 4Gi
          command:
            - /bin/bash
          args:
            - /app/bootstrap.sh
          env:
            - name: CONFIG_PROP
              value: "config-postgres.properties"
          ports:
            - containerPort: 8080
              name: http
          volumeMounts:
            - name: db-config
              mountPath: /app/config/config-postgres.properties
              subPath: config-postgres.properties
            - name: bootstrap
              mountPath: /app/bootstrap.sh
              subPath: bootstrap.sh
      volumes:
        - name: db-config
          configMap:
            name: server-db-config
        - name: bootstrap
          configMap:
            name: server-bootstrap
      restartPolicy: Always
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: server-db-config
  namespace: conductor
data:
  config-postgres.properties: |
    # Servers.
    conductor.grpc-server.enabled=false

    # Database persistence type.
    conductor.db.type=postgres

    spring.datasource.url=jdbc:postgresql://conductor-db.conductor.svc.cluster.local:5432/conductor
    spring.datasource.username=conductor
    spring.datasource.password=conductor

    # Hikari pool sizes are -1 by default and prevent startup
    spring.datasource.hikari.maximum-pool-size=10
    spring.datasource.hikari.minimum-idle=2

    # Elastic search instance indexing is enabled.
    conductor.indexing.enabled=true

    # Transport address to elasticsearch
    conductor.elasticsearch.url=http://elasticsearch.conductor.svc.cluster.local:9200

    # Name of the elasticsearch cluster
    conductor.elasticsearch.indexName=conductor

    conductor.elasticsearch.clusterHealthColor=yellow

    # Load sample kitchen sink workflow
    # loadSample=true
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: server-bootstrap
  namespace: conductor
data:
  bootstrap.sh: |
    #!/bin/bash
    mkdir -p /app/logs
    touch /app/logs/server.log
    /app/startup.sh & tail -f /app/logs/server.log
---
apiVersion: v1
kind: Service
metadata:
  name: conductor-server
  namespace: conductor
spec:
  selector:
    app: conductor-server
  type: ClusterIP
  ports:
    - name: http
      protocol: TCP
      port: 8080

Conductor UI

apiVersion: apps/v1
kind: Deployment
metadata:
  name: conductor-ui
  namespace: conductor
  labels:
    app: conductor-ui
spec:
  selector:
    matchLabels:
      app: conductor-ui
  replicas: 1
  template:
    metadata:
      labels:
        app: conductor-ui
    spec:
      containers:
        - name: conductor-ui
          image: conductor-ui
          env:
            - name: WF_SERVER
              value: "http://conductor-server.conductor.svc.cluster.local:8080/api/"
          ports:
            - containerPort: 5000
              name: conductor-ui
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: conductor-ui
spec:
  ports:
    - port: 80
      targetPort: 5000
      protocol: TCP
  type: NodePort
  selector:
    app: conductor-ui
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: conductor
  name: conductor-web-ui-ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: instance
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: conductor-ui
                port:
                  number: 80

I've deployed Conductor on AWS EKS.

[kishorebanala]

@jasondavindev Awesome, thanks for sharing. Do you mind putting this somewhere with instructions and sending us a PR linking to related section: https://github.com/Netflix/conductor/blob/main/RELATED.md. I believe this would help someone in the community as a reference.

[ZergRushJoe]

I would add a +1 to have ready to go images available

[dongminglei]

If ES7.x has an account password, how to configure it in ConfigMap

[ntk148v]

Hi folks,

I've published Conductor images to Docker Hub:

• https://hub.docker.com/r/kiennt26/netflix-conductor-ui
• https://hub.docker.com/r/kiennt26/netflix-conductor-server
• https://hub.docker.com/r/kiennt26/netflix-conductor-aio

Build script and sample compose files are available here: https://github.com/ntk148v/dockerfiles/tree/master/conductor

[ScottJOster]

Great work on the k8s yaml thanks , can I ask if you encountered errors with beans (metadata dao for eg) and loading context in the server?

It's a well versed issue in here with one fix suggesting adding run time only Postgres version into server gradle but it doesn't seem to work no matter which version I specify!

I get these errors on local docker and k8s when running with compose and postgres yaml command as specified in docs.

Thanks