Add test for basic load balancing (Round Robin) to multicluster tests

[trepel]

Summary

Add test for basic load balancing (Round Robin) to multicluster tests.

Documentation

• https://docs.kuadrant.io/0.8.0/kuadrant-operator/doc/reference/dnspolicy/
• https://docs.kuadrant.io/0.8.0/multicluster-gateway-controller/docs/how-to/multicluster-loadbalanced-dnspolicy/#weighted

Additional context

It is multicluster test, so at least two clusters are required. Start with DNS on AWS (Route 53).

On each cluster

• 1 Gateway
• 1 DNSPolicy for each gateway with load balanced strategy
• Default GEO set to the same in each policy
• No custom weighting

Success Criteria

• All policies get to ready state
• Gateways should have enforced and affected status
• Validate on cloud provider level that the DNS records were created as expected. This might be simpler/more stable than generating requests and validating that both clusters are handling them. It takes time for DNS to propagate changes so it won't be as simple as generating 2 requests and asserting than one was handled by cluster1 and other by cluster2. It might take minutes for DNS to swap clusters.
• only one IP address should be returned, not many (as is the case for simple routing strategy, see the test here: https://github.com/Kuadrant/testsuite/blob/main/testsuite/tests/multicluster/test_multicluster_dns.py)

Follow ups

As part of working on this one can also implement (or create a follow up issues for):

• 10 Gateways on each cluster instead of 1, each Gateway should have at one shared host and one non-shared host
• Test to timeout after 15 minutes if policies still not ready
• Ability to test other cloud providers

[averevki]

@trepel Simple and basic strategies refer to the same test that you are linking here, as far as I understand. The issue that we have with the existing test is it doesn't return addresses in round robin in kubernetes pod, so it can't run in nightlies successfully. This is the problem with kubernetes resolver(caching?) or maybe python dns library

[trepel]

@averevki Not sure if I follow. When I said "simple routing strategy" I meant having .spec.routingStrategy: simple in DNSPolicy. This is what we currently have covered by that test_multicluster_dns test. When I said basic loadbalancing I meant:

spec:
  routingStrategy: loadbalanced
  loadBalancing:
    weighted:
      defaultWeight: 10

which effectively produces Round Robin.

Now I don't understand this fully but I think that only one IP should be returned (not a list) if using having Round Robin configured. If this is indeed the case we can create a test for that if for nothing else.

What we can also do is to check the generated DNS records in cloud provider directly. This would require test suite to have access to cloud provider (AWS likely to be the one) and to configure the DNS stuff properly.

We can keep generating requests for X (not sure for how many) minutes and validating that both clusters were used for handling at least 1 of the generated requests.