Simplify geo and weighted options in DNSPolicy

[maleck13]

What

In the past we worked with OCM which brought the idea of a hub cluster and multiple views of the same gateway. We no longer support OCM but some of that legacy has remained in our API we should simplify this to reflect a policy can only target one gateway and even if it could target multiple they would be in the same cluster.

Currently we have

  loadBalancing:
    weighted:
      defaultWeight: 120
      custom: # <--- New Custom Weights being added
        - weight: 255
          selector:
            matchLabels:
              kuadrant.io/lb-attribute-custom-weight: AWS
    geo: 
      defaultGeo: US #catch all geo         

we then have to set attributes on the gateway in the form of annotations to set these custom weights. This no longer makes sense.

Proposed change is to have the values set directly on the policy and indicated if geo is a default or not

  loadBalancing: # each of these values would be required.
    weight: 100
    geo: U.S
    defaultGEO: true #indicates whether to use this as a the wilcard / catchall geo

• [x] Add RFC https://github.com/Kuadrant/architecture/pull/105
• [ ] Update Kuadrant Opertor with new structure
• [ ] Update Docs with new structure
• [ ] Upadate e2e tests https://github.com/Kuadrant/testsuite/blob/main/testsuite/kuadrant/policy/dns.py

[maleck13]

@mikenairn any additionl thoughts here? If not I can look at putting together and RFC would ideally like this to happen before v1 @philbrookes @maksymvavilov

[maleck13]

@trepel @azgabur This is an important up-coming change we are going to prioritise.

[guicassolato]

@maleck13, do we want this change into v1alpha2, therefore also automatically in v1, of the DNSPolicy API (targeting Kuadrant v1)?

[maleck13]

yes