compose-x / ecs_composex

Manage, Configure and Deploy your services and AWS services and applications from your docker-compose definitions
https://docs.compose-x.io
Mozilla Public License 2.0
158 stars 16 forks source link

[FR] Lookup existing LoadBalancer #687

Closed tmeijn closed 4 months ago

tmeijn commented 10 months ago

Is your feature request related to a problem? Please describe. Hi! If I read the documentation correctly, it's currently not possible to use an existing load balancer.

Maybe there is a limitation I didn't think of why this isn't possible...

Describe the solution you'd like I'd like to use Lookup like there is for other resources to find and existing load balancer.

Describe alternatives you've considered Each Review App provisioning their own ALB.

Additional context The solution I'm thinking about is giving the ability to create Review Apps. Developers would use the existing ALB to expose their Review Apps instead of each Review App provisioning their own ALB. I managed to something similar with the K8S AWS ALB controller + ExternalDNS, but I'd like to experiment with a lower friction solution.

JohnPreston commented 10 months ago

Thanks @tmeijn for the request. This is something that I have been thinking about for a while, waiting to have that FR ;)

In a first version, do you want to have the ability to add

EDIT: that was a question :D

tmeijn commented 10 months ago

@JohnPreston Ah that is a good question! In my K8S POC, ALB controller handled the magic for me. There, it would add listener if it didn't exist and otherwise add a rule to the already existing listener. Also see IngressGroup in the AWS ALB controller docs.

Based on that, I think update a listener -> would be restricted to adding new rules (for ALB only) would give it the best parity, WDYT?

JohnPreston commented 10 months ago

Agreed. Now that LB & Listeners both can have Tags (which reminds me I need to add something to allow Create to set these), we can lookup both. As for rules to add to an existing listener, the trick will be in the priority order. I think a default behaviour of adding it to the "front" of the list is best, as it will be evaluated first. There can't be two default rules so I just need to be certain that won't happen.

Has everything otherwise worked if/when you created a new LB & the rules otherwise?

tmeijn commented 10 months ago

Has everything otherwise worked if/when you created a new LB & the rules otherwise?

To be absolutely honest, I only looked at this tool from documentation perspective to look if it might be a great alternative to the K8S POC and spotted this difference 🙇🏾

As for rules to add to an existing listener, the trick will be in the priority order.

In the controller you can set order explicitly. It's a bit easier though for the Controller, since it has awareness of all the ingresses, while I think when you would use Compose-X it is not centralized. Maybe it would be better to add it as last explicitly defined order? Then you could query the AWS API for the rules list, sort the rule orders and add the new rule as the last priority to that order?

JohnPreston commented 10 months ago

Just to be clear this tool today has nothing to do with K8s so if there is any expectation to use it, might do in the future, but it's meant for ECS workloads.

The controller yeah has the config coming to it and is storing the config "centrally" but so is AWS for the ALB rules. Compoes-X will have to discover the state everytime it's executed. But that's not necessarily a bad thing, especially if for example someone adds stuff manually to the LB.

The rule if not given an ID I think should come first as it's like ACLs, evaluations goes from highest priority (smallest number) to the lowest one.

There also can't be 2 rules with the same conditions and that sort of things.

To mess with bots and scraping, I typically for example always have my Listener forward to containers with at least the path , if not the path + the hostname to accept. That creates the default (last) rule as a 418 I am a teapot. So bots and stuff going around based on IP don't get anything in return, ever.

Will create a branch and submit a first draft of the feature. Sounds good?

EDIT: of course the user will be able to specify the order either with a specific ID (number) or something like "last". But I have had issues in the past with updating the last rule of the listener of a LB. It has to be a perfect change :/