Write ADR for load-balancers

SovereignCloudStack / issues

This repository is used for issues that are cross-repository or not bound to a specific repository.

https://github.com/orgs/SovereignCloudStack/projects/6

2 stars 1 forks source link

Write ADR for load-balancers #227

Open garloff opened 1 year ago

garloff commented 1 year ago

As a KaaS operator, I want to understand exactly what standards my load-balancer must fulfill by reading the relevant ADR.

TODO: Write ADR (according to SCS-0001 standard) to document the requirements tested for in #212.

Definition of Ready:

[ ] User Story is small enough to be finished within one sprint
[ ] User Story is clear and understood by the whole team
[ ] Acceptance criteria are defined
[ ] Acceptance criteria are clear and understood by the whole team

Definition of Done:

[ ] All acceptance criteria are met
[ ] Changes have been reviewed
[ ] CI tests have run successfully
[ ] Documentation has been updated
[ ] Release Notes have been updated

garloff commented 1 year ago

nginx-ingress expects that the single nginx instance on a multi-node cluster works despite externalTrafficPolicy: Local (which means kube-proxy is NOT forwarding connections) on a multi-node cluster. Apparently, nginx-ingress expects the LoadBalancer to detect which nodes are up and load-balance only to the ones where nginx is listening. This is the most efficient way (by avoiding unneeded forwards), but one could question whether loadbalancers should always detect nodes being down. With the default setting of the OpenStack Cloud Controller Manager, loadbalancers do NOT fulfill this: They don't automatically get a health monitor. This is not unreasonable either -- health monitors can always be requested, though unfortunately, this is not standardized but can only be achieved by an OpenStack specific annotation. Which makes things non-portable :-( So we will have to default to health-monitors be enabled ...

garloff commented 1 year ago

TODO:

Investigate why the default to NOT have health monitors was chosen.
Investigate whether opt-out (by specific annotation) is still possible
Compare to other CCMs and their defaults

garloff commented 1 year ago

@garloff and @joshmue to look at OCCM in first week of Jan.

garloff commented 1 year ago

Needs more discussion:

Making traffic flow work (with the help of the health monitor) may result in something half working for the user:
- externalTrafficPolicy: Local results in working traffic
- BUT the client IPs are not preserved!

So: Investigate whether we can address the client IP thing:

proxy protocol (working for http/https like protocols at least)
generic approach using L3 loadbalancer (DSR like), maybe possible with ovn magic ...

jschoone commented 1 year ago

Hi @joshmue. Are we still on track with this?

joshmue commented 1 year ago

The ADR in SovereignCloudStack/standards#169 is pretty much complete. IIRC, the decision that is documented in this record did also match the consensus in a (team?) meeting. Hence, it just needs to be pushed forward according to the according process.

joshmue commented 2 months ago

Refs SovereignCloudStack/standards#648

Will assign @cah-hbaum, please feel free to re-assign me, if I can/should do anything here.