Log collection for applications in K8s cluster

[supereagle]

Requirements for log collection:

• No Invasive: No need to modify applications
• Low Cost: No need to run log collector in EVERY pod
• Friendly: No need to customize K8s

[supereagle]

Overview

Process

1. Create log folder for each pod through Pod initialization Init Script:

   mkdir -f /docker/logs/${APP_NAME}/${POD_NAME}
   rm -rf /apps/logs
   ln -s /docker/logs/${APP_NAME}/${POD_NAME} /apps/logs

2. Create pod by spec with env and mount

   spec:
   containers:
   - env:
   - name: POD_NAME
     valueFrom:
       fieldRef:
         apiVersion: v1
         fieldPath: metadata.name
   volumeMounts:
   - mountPath: /docker/logs/
     name: applogs
   volumes:
   - hostPath:
     path: /apps/logs/
   name: applogs

3. Startup application in pod and it will output their logs into /apps/logs as before
4. Log collector in each K8s node will collect app logs from host path /apps/logs/, and reports the logs to log server