octavia-ingress-controller / occm : Different Octavia behavior from Ingress vs svctype LoadBalancer

[ReggieCarey]

/kind bug

What happened:

When I deploy an ingress object into the cluster, I get a loadbalancer in Openstack that has an attached Floating ip and security Group. When I change the type of a service from NodePort to LoadBalancer, a load balancer is created with an attached Floating ip but no security group is created.

Is this expected behavoir?

I'm trying to deploy kubeflow but it uses Istio and Istio uses Gateway and VirtualService in preference to Ingress.

As a result I cannot access Kubeflow dashboards.

What you expected to happen:

I expected that changing the type of a Service from NodePort to LoadBalancer would result in a working Octavia LoadBalancer to include FIP, and Security Groups.

Said security groups get created when the loadbalancer is initiated by an Ingress deployed to the cluster.

How to reproduce it:

Deploy Kubeflow. Change the istio-ingressgateway service type from NodePort to LoadBalancer.

Also tried cloud-provider-openstack/docs/openstack-cloud-controller-manager/expose-applications-using-loadbalancer-type-service.md

and this failed as well

Anything else we need to know?:

Environment:

Octavia Ingress Controller: docker.io/k8scloudprovider/octavia-ingress-controller:latest"

App                   Version   Status  Scale  Charm                 Channel   Rev  Exposed  Message
containerd            go1.13.8  active      6  containerd            stable    200  no       Container runtime available
easyrsa               3.0.1     active      1  easyrsa               stable    441  no       Certificate Authority connected.
etcd                  3.4.5     active      3  etcd                  stable    655  no       Healthy with 3 known peers
flannel               0.11.0    active      6  flannel               stable    619  no       Flannel subnet 10.1.15.1/24
grafana                         active      1  grafana               stable     51  yes      Ready
kubernetes-master     1.21.11   active      2  kubernetes-master     stable   1106  no       Kubernetes master running.
kubernetes-worker     1.21.11   active      4  kubernetes-worker     stable    838  yes      Kubernetes worker running.
openstack-integrator  xena      active      1  openstack-integrator  stable    204  no       Ready
prometheus                      active      1  prometheus2           stable     25  no       Ready
telegraf                        active      6  telegraf              stable     44  no       Monitoring kubernetes-worker/2 (source version/commit 26e531a)

[jichenjc]

not fully understand the question , are you saying the sec group is not auto created so the network has connection issue?

do you happen to check manage-security-groups in https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/octavia-ingress-controller/using-octavia-ingress-controller.md ?

[ReggieCarey]

Yes that is correct. The security group in openstack does not get created with LoadBalancer typed Services but does get created with ingresses

Sent from my iPhone

On Apr 14, 2022, at 22:13, ji chen @.***> wrote:

 not fully understand the question , are you saying the sec group is not auto created so the network has connection issue?

do you happen to check manage-security-groups in https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/octavia-ingress-controller/using-octavia-ingress-controller.md ?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[jichenjc]

@lingxiankong I think it's a gap between lbaas and octavia?

https://github.com/kubernetes/cloud-provider-openstack/blob/master/pkg/openstack/loadbalancer.go#L3239 is for lbaas and it created the sec group

but if it's octavia then we go https://github.com/kubernetes/cloud-provider-openstack/blob/master/pkg/openstack/loadbalancer.go#L1951

and I didn't see sec gropu related functions https://github.com/kubernetes/cloud-provider-openstack/blob/bfe44893dbd37334f2f28b387027ea81d1c1dbb5/pkg/openstack/loadbalancer.go#L1778

so is it on purpose?

[ReggieCarey]

Just a comment on how awesome this open source group is. You guys are so responsive!

@jichenjc to your question concerning manage-security-groups. Yes I have that parameter set to true in my configuration:

I have yet to include Go to my list of languages, but it's mostly a matter of picking up the syntax.

Looking through the code snippets, I'm not sure I understand the refs to the two code lines. The first deals with removing Security Groups if ManageSecurityGroups is true and the second deals with short circuiting LB creation if favor of OctaviaLoadBalancer creation if lbaas.opts.UseOctavia is true:

    if lbaas.opts.UseOctavia {
        return lbaas.ensureOctaviaLoadBalancer(ctx, clusterName, apiService, nodes)
    }

The false path leads to possible security group creation at https://github.com/kubernetes/cloud-provider-openstack/blob/5a5030e83fd72838cddb075bef19e46ba999676e/pkg/openstack/loadbalancer.go#L2415

However, a cursory read through the true path shows no similar call to lbaas.ensureSecurityGroup(..) as we see on line 2416.

[jichenjc]

@ReggieCarey yes, that's similar question that I am asking @lingxiankong 's opinion likely the function is missing due to we first support neutron lbaas then octavia and sec group part is not included ,if so we can treat this as bug and add such support