haproxytech / helm-charts

Helm chart for HAProxy Kubernetes Ingress Controller
Apache License 2.0
148 stars 118 forks source link

peers are not in sync post deployment using helm charts #262

Closed Govind10g closed 6 days ago

Govind10g commented 6 days ago

Hi Team,

Hi Team,

I m running multiple HA proxy pod and trying to understand if HA proxy pods are in sync using peers concept. When trying to execute below show command, I m not able to find anything :-

/ # echo “show peers” | socat stdio /var/run/haproxy.sock

though peers info is set in the config but no output. below is my configuration :-

## HAProxy daemon configuration
# ref: https://www.haproxy.org/download/2.6/doc/configuration.txt
config: |
  global
    stats socket /var/run/haproxy.sock mode 600
    log stdout format raw local0
    maxconn 1024

  defaults
    log global
    timeout client 60s
    timeout connect 60s
    timeout server 60s

  frontend jms
       mode http
       bind *:80
       default_backend jms

  peers haproxy-peers
      # peer <peer_name> <ip:port>
      peer i-012345 haproxy-1.ha-proxy.svc.cluster.local:1024
      peer i-012367 haproxy-2.ha-proxy.svc.cluster.local:1024 

  backend jms
        mode http
        balance leastconn
        stick-table type string len 256 size 200k expire 120m peers haproxy-peers
        stick on url_param(room) table jms
        hash-type consistent

        server jitsi-1 10.22.33.22:80 id 88
        server jitsi-2 10.33.11.34:80 id 72

On further debugging, I m seeing Alert and warning in the haproxy command output, mentioned below :-

/ # 
/ #  haproxy -db -f /etc/haproxy/haproxy.cfg -L i-012367
[NOTICE]   (27) : haproxy version is 2.9.6-9eafce5
[ALERT]    (27) : Binding [/etc/haproxy/haproxy.cfg:21] for peers section i-012367: cannot bind socket (Address not available) for [172.25.78.105:1024]
[ALERT]    (27) : [haproxy.main()] Some protocols failed to start their listeners! Exiting.
/ # 
/ #
/ # haproxy -f /etc/haproxy/haproxy.cfg -c
[NOTICE]   (29) : haproxy version is 2.9.6-9eafce5
[WARNING]  (29) : config : Removing incomplete section 'peers haproxy-peers' (no peer named 'haproxy-1-7d79b7d54-mdvn5').
Warnings were found.
/ # 
/ #
/ # echo "show peers haproxy-peers" | socat stdio /var/run/haproxy.sock
2024/09/18 11:20:35 socat[35] E UNIX-CLIENT:/var/run/haproxy.sock: Connection refused
/ # 

Could you please guide me what is wrong here ?

Also how we can check the status of haproxy as I’m not able to see anything with using haproxy or haproxy.service in the pod.

Govind10g commented 6 days ago

Hi @dkorunic -

Can you help me on this issue ?

dkorunic commented 6 days ago

Hi, this issue is not really related to Helm Chart. That aside, you are trying to bind peer IP which is not local IP address. More specific:

Defines a peer inside a peers section.
If <peername> is set to the local peer name (by default hostname, or forced
using "-L" command line option or "[localpeer](https://www.haproxy.com/documentation/haproxy-configuration-manual/latest/#localpeer)" global configuration setting),
HAProxy will listen for incoming remote peer connection on the provided
address.  Otherwise, the address defines where to connect to in order to join
the remote peer, and <peername> is used at the protocol level to identify and
validate the remote peer on the server side.

During a soft restart, local peer address is used by the old instance to
connect the new one and initiate a complete replication (teaching process).

It is strongly recommended to have the exact same peers declaration on all
peers and to only rely on the "-L" command line argument or the "[localpeer](https://www.haproxy.com/documentation/haproxy-configuration-manual/latest/#localpeer)"
global configuration setting to change the local peer name. This makes it
easier to maintain coherent configuration files across all peers.

Please refer to the HAProxy configuration reference for more information.

Govind10g commented 6 days ago

Hi Thank you so much for the reply.

If you see my peers configuration, as mentioned below -

  peers haproxy-peers
      # peer <peer_name> <ip:port>
      peer i-012345 haproxy-1.ha-proxy.svc.cluster.local:1024
      peer i-012367 haproxy-2.ha-proxy.svc.cluster.local:1024 

I m using service url to reach to my HA-proxy pod 1 and pod 2 respectively and my understanding is that it will try to sync both pods of HA proxy pod and I can't provide the IP which is dynamic in the case of kubernetes.

then my doubt is if I m passing the service url, it should resolve to the Ha proxy pod right instead of service resolving to the Cluster-IP of service ?

Govind10g commented 6 days ago

How do you suggest to write peers in the ha proxy config in Kubernetes deployment ?