[bitnami/apisix] failed on restart of container

maipal-c commented 2 days ago

Name and Version

bitnami/apisix:3.5.0

What architecture are you using?

arm64

What steps will reproduce the bug?

1 install the chart using the values below

simulate restart (keep memory requirement low so you will see OOM killed and the container will get recreated within the pod.)

when the container gets recreated (not in the new pod) it fails to start with the following error

"2024/10/06 13:26:22 [emerg] 1#1: bind() to unix:/usr/local/apisix/logs/worker_events.sock failed (98: Address already in use)"

Are you using any custom parameters or values?

default values with coraza wasm plugin enabled on the control plane & data plane

dataPlane:
  extraConfig:
        wasm:
           plugins:
              - name: coraza-filter
                priority: 7999
                file: /tmp/wasm-plugins/coraza-proxy-wasm.wasm
     extraVolumes:
        - name: wasm-plugins
          emptyDir: {}
     extraVolumeMounts:
        - name: wasm-plugins
          mountPath: "/tmp/wasm-plugins"
     initContainers:
        - name: attach-wasm-plugins
          image: busybox
          securityContext:
             capabilities:
                drop:
                   - ALL
             privileged: false
             runAsUser: 1001
             runAsGroup: 1001
             runAsNonRoot: true
             readOnlyRootFilesystem: true
             allowPrivilegeEscalation: false
             seccompProfile:
                type: RuntimeDefault
          volumeMounts:
             - name: wasm-plugins
               mountPath: "/tmp/wasm-plugins"
          command:
             - "sh"
             - "-c"
             - |
                cd /tmp/wasm-plugins ;
                wget https://github.com/corazawaf/coraza-proxy-wasm/releases/download/0.5.0/coraza-proxy-wasm-0.5.0.zip ;
                unzip coraza-proxy-wasm-0.5.0.zip ;
                rm coraza-proxy-wasm-0.5.0.zip

controlPlane: <same_as_above>

What is the expected behavior?

on recreation of container, it should start normally.

What do you see instead?

"2024/10/06 13:26:22 [emerg] 1#1: bind() to unix:/usr/local/apisix/logs/worker_events.sock failed (98: Address already in use)"

Additional information

same issue on apisix github repo, here

possible fix -

In Apisix's official helm chart they have a lifecycle hook -

lifecycle:
  preStop:
    exec:
      command:
        - /bin/sh
        - -c
        - "sleep 30"

maybe we should also use a prestart hook that will run "rm /usr/local/apisix/logs/worker_events.sock" or maybe use the same apisix approach

carrodher commented 2 days ago

Thank you for bringing this issue to our attention. We appreciate your involvement! If you're interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

maipal-c commented 2 days ago

hey @carrodher,

which approach i should use? preStop "sleep 30" OR postStart "rm /usr/local/apisix/logs/worker_events.sock" OR maybe preStop "rm /usr/local/apisix/logs/worker_events.sock". i'll be happy to contribute that small code segment.

currently i am testing preStop "sleep 20" for my installation...

drop in the approach, i'll submit PR

thank you

bitnami / charts