KafkaSinkCluster scram_over_mtls - initial integration tests

[rukai]

This PR pulls the majority of the changes out of https://github.com/shotover/shotover-proxy/pull/1651 since they require no code changes to land.

To support the new test cases, this PR first extends the kafka connection abstraction. This abstraction abstracts over both the CPP driver and java driver. For this PR we can get away with just supporting the java driver since the cpp driver doesnt support SCRAM and so we cant use it for this test anyway. This PR adds:

• a KafkaAdmin::describe_topic method
  • we only care about checking if the describe request succeeded or failed, so we dont actually need to return any info on the topic. But we follow the structure of a proper describe_topic method so that it can be extended to a full implementation in the future if needed.
• a KafkaAdmin::create_acls method
  • ACL's have a lot of different enums they need, these are all defined here as well.
  • Some of the enums have variants that arent actually used for acl creation, so I have not included those variants in the enums.
  • Usually for these kinds of enum's the matchs in create_acls would go on methods on the enum. e.g. we would call something like acl.resource_type.to_field_name(). But I've not done this here, since that is a java specific implementation detail that I wouldnt want living on the generic ResourceType enum.
  • The actual implementation is largely java interop soup, dont stress about it too much.

Then this PR implements new test cases on top of the new connection abstraction methods:

• setup_basic_user_acls utilizes KafkaAdmin::create_acls to configure basic_user with the ability to describe topics.
• assert_topic_creation_is_denied_due_to_acl is then defined as an important correctness assertion, to prove that we are not mixing up tokens. It tests not just that the request returns an error but that the action of the request was not performed. i.e. no topic is created.

And then finally we run the test cases in various configurations in cluster_sasl_scram_over_mtls_single_shotover:

• Usually our integration tests just use a single user configured as a super user. However here we need to test multiple users with different access levels, so we setup a super_user and a basic_user in the docker-compose.yaml
  • this meant we also changed the user name in cluster_sasl_scram_over_mtls_multi_shotover to super_user
• assert_topic_creation_is_denied_due_to_acl is called with and without a concurrent super user connection
• Further test cases will be added in https://github.com/shotover/shotover-proxy/pull/1651

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #1652 will not alter performance

_{Comparing rukai:scram_over_mtls_tests_prereq (1cce034) with main (d9e441f)}

Summary

✅ 37 untouched benchmarks