bitnami / charts

Bitnami Helm Charts
https://bitnami.com
Other
8.81k stars 9.1k forks source link

bitnami/postgres-ha pgpool crash issues #27920

Closed DeprecatedLuke closed 1 month ago

DeprecatedLuke commented 1 month ago

Name and Version

bitnami/postgres-ha:15.5.0

What architecture are you using?

amd64

What steps will reproduce the bug?

  1. default configuration with increased resource limits and changed passwords.
  2. k3s v1.30 cluster, ceph rbd file backend, calico vxlan with hostendpoint configured.
  3. wait until the bug occurs, setting low limits seems to cause the bug to occur much faster.

Are you using any custom parameters or values?

  postgresql-ha:
    clusterDomain: custom.cluster
    global:
      postgresql:
        database: gitea
        password: anything
        username: gitea
    enabled: true
    postgresql:
      replicaCount: 2
      upgradeRepmgrExtension: false
      repmgrPassword: anything
      postgresPassword: anything
      password: anything
    pgpool:
      adminPassword: anything
      replicaCount: 1
      resources:
        limits:
          cpu: 375m
          ephemeral-storage: 1Gi
          memory: 512Mi
        requests:
          cpu: 250m
          ephemeral-storage: 50Mi
          memory: 256Mi
    primary:
      persistence:
        size: 10Gi

What is the expected behavior?

pgpool restarting properly.

What do you see instead?

After a month or two, pgpool service gets stuck in a crashloop with no logs being output to the console. Deleting the pod is the only way to fix it.

Additional information

Pgpool is the only pod that is showing this behavior out of around a hundred other pods, increased limits seemed to somewhat mitigate the issue, seems like an early crash at the entry point (before any logging occurs).

Unfortunately, there are pretty much no logs to provide as there aren't any except restart counter going up and replicating it is very inconsistent.

carrodher commented 1 month ago

The issue may not be directly related to the Bitnami container image/Helm chart, but rather to how the application is being utilized, configured in your specific environment, or tied to a specific scenario that is not easy to reproduce on our side.

If you think that's not the case and are interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

Suppose you have any questions about the application, customizing its content, or technology and infrastructure usage. In that case, we highly recommend that you refer to the forums and user guides provided by the project responsible for the application or technology.

With that said, we'll keep this ticket open until the stale bot automatically closes it, in case someone from the community contributes valuable insights.

DeprecatedLuke commented 1 month ago

This issue seems to be only present in postgres-ha chart, even running standalone pgpool does not exhibit same behavior. The only thing that comes to mind are policies, but those are relatively standard across all charts.

github-actions[bot] commented 1 month ago

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

github-actions[bot] commented 1 month ago

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.