SocketCluster / socketcluster

Highly scalable realtime pub/sub and RPC framework
https://socketcluster.io
MIT License
6.15k stars 314 forks source link

Kubernetes - Use initContainer instead of another container as source volume #464

Open JCMais opened 5 years ago

JCMais commented 5 years ago

It's better doing it that way because it's not necessary to wait for the files to become available (as seen here: https://github.com/SocketCluster/socketcluster/blob/5226c2a/sample/dockerwait.js).

Example of how it would work:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-project-scc
spec:
  replicas: 2
  revisionHistoryLimit: 2
  template:
    metadata:
      labels:
        app: my-project-scc
    spec:
      initContainers:
        - name: src-container
          image: myuser/sourceimage
          command:
            - cp
            - '-a'
            - /usr/src/.
            - /usr/dest/
          volumeMounts:
            - mountPath: /usr/dest
              name: src-volume
          resources:
            limits:
              cpu: 10m
              memory: 50Mi
            requests:
              cpu: 10m
              memory: 50Mi
      containers:
        - name: socketcluster
          image: 'socketcluster/socketcluster:v14.3.0'
          ports:
            - containerPort: 8000
          env:
            - name: SCC_STATE_SERVER_HOST
              value: my-project-scc-state
            - name: SCC_INSTANCE_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: SOCKETCLUSTER_WORKER_CONTROLLER
              value: /usr/src/app/worker.js
            - name: SOCKETCLUSTER_MASTER_CONTROLLER
              value: /usr/src/app/server.js
            # ... other envs
          resources:
            limits:
              cpu: 100m
              memory: 360Mi
            requests:
              cpu: 100m
              memory: 360Mi
          livenessProbe:
            httpGet:
              path: /health-check
              port: 8000
            initialDelaySeconds: 300
            timeoutSeconds: 5
          volumeMounts:
            - mountPath: /usr/src/app
              name: src-volume
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: app
                    operator: In
                    values:
                      - my-project-scc
              topologyKey: kubernetes.io/hostname
      volumes:
        - name: src-volume
          emptyDir: {}

The container is only started when the init container has finished. I was using that until recently, having both as normal containers can cause some problems, and having to set the wait timeout to bigger values each time newer dependencies are added.

Recently I've ditched this deployment in favor of building my own socketcluster image with the correct dependencies already installed, since copying node_modules from the image to the volume was taking too long (almost 20 mins). But it's here as a suggestion.

jondubois commented 5 years ago

@JCMais Thanks for pointing this out. I'll investigate this solution.

jondubois commented 5 years ago

@JCMais I finally got some time to test your approach on GKE and it worked perfectly and it's definitely much cleaner so I'll try to push with an update soon.

JCMais commented 5 years ago

Glad it worked 😄

happilymarrieddad commented 5 years ago

Yea, I need to start using initContainer. I haven't started using it yet. I was going to on my next project. Thanks for the example!