kubernetes-sigs / aws-load-balancer-controller

A Kubernetes controller for Elastic Load Balancers
https://kubernetes-sigs.github.io/aws-load-balancer-controller/
Apache License 2.0
3.93k stars 1.46k forks source link

Create option to reuse an existing ALB instead of creating a new ALB per Ingress #298

Closed julianvmodesto closed 3 years ago

julianvmodesto commented 6 years ago

I read in this comment https://github.com/coreos/alb-ingress-controller/issues/85#issuecomment-299504058 that host-based routing was released for AWS ALBs shortly after ALB Ingress Controller was released.

It would be pretty cool to have an option to reuse an ALB for an Ingress via annotation -- i'd be interested in contributing towards this, but I'm not sure what's needed to make this feasible.

ffjia commented 4 years ago

IAM Role for Service Accounts @kirkdave did you make the ALB ingress IRSA work in EKS?

kirkdave commented 4 years ago

@ffjia It works with IRSA if you either build the Docker image from the branch or, as I did, use the image that @M00nF1sh created in #914 - m00nf1sh/aws-alb-ingress-controller:v1.2.0-alpha.2

rajeshwrn commented 4 years ago

I was came across the same kind of requirement.

Have to deploy 20+ services in kubernetes in AWS Fargate profile. Since the fargate is not support NLB as of now the only option is ALB. But each deployment created new Alb. This needs 20+ public ips and also more cost on alb.

I achieved the solution with two ingress controller, alb ingress and nginx ingress.

Nginx will be the target for alb with port 80. And application services running in cluster with different port and namespaces will communicate nginx.

I have documented my solution. I think it will help your requirement.

https://github.com/rajeshwrn/alb-nginx-controller

MXClyde commented 4 years ago

Any timelines on when this functionality will be part of a stable release?

astrived commented 4 years ago

@MXClyde we are doing the final phase of testing for the new aws-loadbalancer-controller and the RC image is here https://github.com/kubernetes-sigs/aws-alb-ingress-controller/releases/tag/v2.0.0-rc5. You can find details here https://github.com/kubernetes-sigs/aws-alb-ingress-controller/tree/v2_ga. Stay tuned for What's new coming soon !

astrived commented 3 years ago

We have published the v2.0.0 release at https://github.com/kubernetes-sigs/aws-load-balancer-controller/releases/tag/v2.0.0 The documents have been updated https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html and https://docs.aws.amazon.com/eks/latest/userguide/load-balancing.html and the blog post is here https://aws.amazon.com/blogs/containers/introducing-aws-load-balancer-controller/

haalcala commented 3 years ago

Can you guys improve your documentation system like how mongodb or elasticsearch does it? Like in a way that I can select which version of the library/system documentation I want to view? I mean, can you not overwrite the published url with the latest one, coz I have to maintain some old versions, coz it gets confusing trying remember what I did before (with 1.1.4 for example) and looking at version 2.

angapov commented 2 years ago

It seems this is still not implemented. Any plans for supporting it in future?

GoodMirek commented 2 years ago

I am quite sure this feature is already implemented, I used it recently. Look at https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/ingress/annotations/#ingressgroup

angapov commented 2 years ago

Oops, sorry, my bad. I actually thought about reusing existing ALB created outside of Ingress controller (sharing the ALB between multiple EKS clusters). But nevermind.

Tzrlk commented 2 years ago

I've been thinking to try manually adding the tags that the ingress controller uses to identify the ALBs it creates across restarts, so the ALB can be created outside kubernetes and simply controlled by the controller. The benefits of this approach are that I wouldn't have to give the controller permission to create ALBs, only target groups, etc. and could reduce the complexity of my k8s config significantly by configuring the obvious stuff up-front (http -> https redir, ssl cert, etc.) It'd be great if the devs can confirm whether that approach is currently possible, otherwise when I get back to that area of my current project, I'll give it a spin, and report back.

vprus commented 2 years ago

@Tzrlk : just to clarify, you can configure these things with ALB controller. The alb.ingress.kubernetes.io/certificate-arn annotation allows to set SSL certificate, the alb.ingress.kubernetes.io/ssl-policy one sets SSL policy and to redirect HTTP you do something like

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
alb.ingress.kubernetes.io/ssl-redirect: '443'

I agree that reusing an existing balancer might be a nice feature, but most of configurability was added to the ALB controller.

Tzrlk commented 2 years ago

@vprus Indeed I've implemented that method previously. Looking through more of the issues, @angapov's requirement is answered as part of #228 with the release of TargetGroupBindings. In fact, the method I was suggesting above to hack it into place appears to have worked for one of the last commenters there.

CPWu commented 2 years ago

Maybe its me but it didn't work as intended for me. I created an ingress resource and it instantiated an alb, I created a secondary ingress and specified the group name and i did not see that a record was created in the hosted zone or in the rules of the alb's listener.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-bye-ingress
  namespace: hello-world
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/load-balancer-name: hello-world-loadbalancer
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/healthcheck-path: /
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/group.name: test
spec:
  rules:
  - host: helloworld.url.test.ai
    http:
      paths: 
      - backend:
          serviceName: hello-world-service
          servicePort: 80
  - host: byeworld.url.test.ai
    http:
      paths: 
      - backend:
          serviceName: bye-world-service
          servicePort: 80
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: secondary-ingress
  namespace: hello-world
  annotations:
    alb.ingress.kubernetes.io/group.name: test
spec:
  rules:
  - host: this.url.test.ai
    http:
      paths: 
      - backend:
          serviceName: this-world-service
          servicePort: 80
thunder-spb commented 2 years ago

@CPWu What's in alb ingress controller logs? and what's in external-dns logs?

marcosdiez commented 2 years ago

To whom it may concern, I created a PR that partially solves this: https://github.com/kubernetes-sigs/aws-load-balancer-controller/pull/2655

The idea is that you still generate your ALB, the alb rules and the target group with terraform (or whatever else) but now it's possible for the aws load balancer controller to locate the target group via it's name (which happens to be unique).

divvy19 commented 2 years ago

I was able to resolve this using below steps :- Add below tags in the already existing ALB -- 1) ingress.k8s.aws/resource --> LoadBalancer 2) ingress.k8s.aws/stack --> exisitingalb 3) elbv2.k8s.aws/cluster --> name of your cluster

Once the above step is done add the below annotation mapping to your ingress.yaml file

alb.ingress.kubernetes.io/group.name: "exisitingalb"

Save and apply this config using kubectl apply -f ingress.yaml.

This should use the existing load balancer as an ingress resource to provision the alb. All the magic happens via tags

carlosrochap commented 2 years ago

Reference in

This didn't work for me. If you change the ingress name the existing ALB gets deleted and a new ALB is provisioned. Is there currently no way to re-use an existing ALB or make sure you get the same domain?.

Are there any plans to support this?

GoodMirek commented 2 years ago

This didn't work for me. If you change the ingress name the existing ALB gets deleted and a new ALB is provisioned.

@carlosrochap Did you examine the AWS loadbalancer controller logs to find more details why is it happening?

geastman3 commented 1 year ago

@divvy19 or anyone else that knows...if the ingress is deleted, does the controller try to delete the ALB? We can remove the permissions from the controller to be able to create and delete the ALB but I would also be curious if this floods the controller logs with nonstop exceptions while the controller tries to reconcile the deleted ingress object and delete the ALB?

warnerm8 commented 1 year ago

This blog looks to be related:

"TargetGroupBinding is a custom resource managed by the AWS Load Balancer Controller. It allows you to expose Kubernetes applications using existing load balancers. A TargetGroupBinding resource binds a Kubernetes Service with a load balancer target group. When you create a TargetGroupBinding resource, the controller automatically configures the target group to route traffic to a Service."