Added Aqua enforcer Add On

[BaruchBilanski]

Issue #, if available:

Description of changes: Added Aqua enforcer Add On

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[elamaran11]

@BaruchBilanski Checking status of Aqua enforcer looks good to me. We will move to next step of validation.

@mikemcd3912 @Pjv93 Lets take this on priority to validate AquaSec for Conformance.

[mikemcd3912]

Based upon our testing the functional tests are failing to report status on Bottlerocket OS compute, and are unable to deploy the solution's pods successfully on ARM based compute. @BaruchBilanski Can you please confirm whether these options are currently supported and if not are there plans in your roadmap to support Bottlerocket and ARM in the future?

[mikemcd3912]

Current Behavior in our testing environments:

• VMware (K8s 1.28, Bottlerocket 1.15): Pods report a running state, no error events for readiness or liveness probes. Test Job fails after 5m aqua-vmware-tester-logs.txt aqua-vmware-pod-describe.txt aqua-vmware-logs.txt aqua-vmware-tester-describe.txt

• Baremetal (K8s 1.27, Ubuntu 20.04.6 LTS): Pods report a running state, no error events for readiness or liveness probes. Test job completes successfully

• Outposts (K8s 1.27, Amazon Linux 2): Pods eventually report a ready state but are experiencing Warning Events for Unhealthy liveness and readiness probes due to refused connections. Test Job completes successfully in 90s aqua-outpost-pod-describe.txt aqua-outpost-logs.txt

• EKS (K8s 1.28, x86 Amazon Linux 2): Pods report a running state and later fail readiness probe with status 503, Test job completes successfully aqua-eks-al2-x86-pod-describe.txt aqua-eks-al2-x86-logs.txt

• EKS (K8s 1.28, x86 Bottlerocket): Pods report a running state, no error events for readiness or liveness probes. Test job fails after 5m aqua-eks-br-x86-pod-describe.txt aqua-eks-br-x86-tester-logs.txt aqua-eks-br-x86-logs.txt aqua-eks-br-x86-tester-describe.txt

• EKS (K8s 1.28, ARM Bottlerocket 1.19): Pods do not deploy due to image pull error aqua-eks-br-arm-pod-describe.txt

[mikemcd3912]

Looks like the latest update has made some progress, but we are still experiencing issues with the ARM image pull:

• VMware (K8s 1.28, Bottlerocket 1.15): Pods deploy and reach a ready state, tester completes successfully

• Baremetal (K8s 1.27, Ubuntu 20.04.6 LTS): Pods deploy and reach a ready state, tester completes successfully

• Outposts (K8s 1.27, Amazon Linux 2): Pods deploy and reach a ready state, tester completes successfully

• EKS (K8s 1.28, x86 Amazon Linux 2): Pods deploy and reach a ready state, tester completes successfully

• EKS (K8s 1.28, x86 Bottlerocket): Pods deploy and reach a ready state, tester completes successfully

• EKS (K8s 1.28, ARM Bottlerocket 1.19): Pods do not deploy due to image pull error. Message reads "Failed to pull image "registry.aquasec.com/enforcer:2022.4.460": rpc error: code = NotFound desc = failed to pull and unpack image "registry.aquasec.com/enforcer:2022.4.460": no match for platform in manifest: not found"

[mikemcd3912]

Thanks for the additional update! It looks like that has solved our image pull issue for ARM, but we're still getting stuck on the tester not connecting on that environmnet. All other environmnets are unaffected by the change and still run as expected:

• VMware (K8s 1.28, Bottlerocket 1.15): Pods deploy and reach a ready state, tester completes successfully

• Baremetal (K8s 1.27, Ubuntu 20.04.6 LTS): Pods deploy and reach a ready state, tester completes successfully

• Outposts (K8s 1.27, Amazon Linux 2): Pods deploy and reach a ready state, tester completes successfully

• EKS (K8s 1.28, x86 Amazon Linux 2): Pods deploy and reach a ready state, tester completes successfully

• EKS (K8s 1.28, x86 Bottlerocket): Pods deploy and reach a ready state, tester completes successfully

• EKS (K8s 1.28, ARM Bottlerocket 1.19): Pods deploy and report ready, Tester fails to connect to pods