search

haproxytechblog / ingress-controller-external-example

An example of running the HAProxy Kubernetes Ingress Controller in external mode.
25 stars 8 forks source link

Setup works with vagrant, but unable to run it on openstack machines. #1

Closed rsingh2411 closed 1 year ago

rsingh2411 commented 3 years ago

The tutorials works fine when running on vagrant. After replicating it on openstack VMs.The traffic is not going from openstack ingress VM to cluster.

all check points are working as mentioned in the example,

` sudo birdc show protocols BIRD 1.6.8 ready. name proto table state since info bgp1 BGP master up 2021-08-21 Established bgp2 BGP master up 2021-08-21 Established bgp3 BGP master up 2021-08-21 Established kernel1 Kernel master up 2021-08-20 device1 Device master up 2021-08-20

sudo birdc show protocols BIRD 1.6.8 ready. name proto table state since info bgp1 BGP master up 2021-08-21 Established bgp2 BGP master up 2021-08-21 Established bgp3 BGP master up 2021-08-21 Established kernel1 Kernel master up 2021-08-20 device1 Device master up 2021-08-20

sudo birdc show route protocol bgp2 BIRD 1.6.8 ready. 172.16.251.128/26 via 192.168.0.22 on ens3 [bgp2 2021-08-21] (100) [i] sudo birdc show route protocol bgp3 BIRD 1.6.8 ready. 172.16.216.0/26 via 192.168.0.24 on ens3 [bgp3 2021-08-21] (100) [i] sudo birdc show route protocol bgp1 BIRD 1.6.8 ready. 172.16.137.0/26 via 192.168.0.5 on ens3 [bgp1 2021-08-21] * (100) [i]

sudo calicoctl node status Calico process is running. IPv4 BGP status +--------------+-------------------+-------+------------+-------------+ | PEER ADDRESS | PEER TYPE | STATE | SINCE | INFO | +--------------+-------------------+-------+------------+-------------+ | 192.168.0.21 | global | up | 2021-08-21 | Established | | 192.168.0.22 | node-to-node mesh | up | 2021-08-20 | Established | | 192.168.0.24 | node-to-node mesh | up | 2021-08-20 | Established | +--------------+-------------------+-------+------------+-------------+

IPv6 BGP status No IPv6 peers found.

Only difference is route command output on ingress node route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface default _gateway 0.0.0.0 UG 100 0 0 ens3 169.254.169.254 _gateway 255.255.255.255 UGH 100 0 0 ens3 172.16.137.0 192.168.0.5 255.255.255.192 UG 0 0 0 ens3 172.16.216.0 192.168.0.24 255.255.255.192 UG 0 0 0 ens3 172.16.251.128 192.168.0.22 255.255.255.192 UG 0 0 0 ens3 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3

There is an extra interface enp0s3: in virtualbox set up for each VM whereas openstack VMs have only ens3 interface.

Is there something more which needs to be configured.

NickMRamirez commented 3 years ago

I don't think it should be a problem to have only one interface. Does the HAProxy Stats page, which listens at port 1024, show the servers? You can also check the file /tmp/haproxy-ingress/etc/haproxy.cfg to see if the servers were added to the backend.

If you are running the ingress controller as a service, check that it has no errors with sudo systemctl status haproxy-ingress

belfhi commented 2 years ago

I'm having the same issue. The first problem was that the ipip traffic was blocked, and I had to add an protocol: 4 security group. Now my calico network works in the k8s cluster but I still can't get traffic from my BGP peer (the loadbalancer) into the cluster. It seems my responses are blocked by Openstack and src / desc check needs to be disabled but I have no idea why?

belfhi commented 2 years ago

@rsingh2411 I got it to work with a lot of help from the guys in the Calico Slack :kissing: You need to disable port security with openstack port set --disable-port-security. This disables the src/dst checks that prevents the packets from coming back.