Verify provisioned clusters pass the EKS CIS benchmark

[mogul]

User Story

In order to give auditors confidence that provisioned EKS clusters are following best-practices, we should be able to demonstrate that a provisioned cluster can pass the CIS EKS benchmark.

Acceptance Criteria

• [x] GIVEN I have installed the tree kubectl plugin \ WHEN I use kubectl tree on pods and nodes \ THEN I see resources that containing scanning results are present
• [x] WHEN I run kubectl get CISKubeBenchReport <nodename> -o wide \ THEN I see a report indicating no tests failed
• [x] WHEN I run kubectl get CISKubeBenchReport <nodename> -o yaml \ THEN I see a detailed report
• [x] WHEN I run make clean build up demo-up demo-test \ THEN I see that there is a test for a CISKubeBenchReport with zero FAIL results

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

See also this GSA ISE hardening guide for EKS

Security Considerations (required)

This change will ensure that any new deployment of the eks-brokerpak will only deploy CIS-compliant instances of AWS EKS. This will bolster confidence in the configuration of the EKS instances we create.

Sketch

1. Install the Aquasec starboard-operator
2. Add lines at the end of the tests that check that the AWS EKS CIS benchmark had zero FAIL results
3. Document how someone can check these reports on any existing instance

Note that AWS Security Hub can ingest kube-bench results. We may want to set this up if it turns out that we need to continuously report on existing instances, but it's probably out of scope for this story. Let's wait to see if it's required, and write that separate story when it's time.

[mogul]

I just noticed this:

The EKS Benchmark is applicable to EC2 nodes (both managed and self managed) where you’re responsible for security configurations of Kubernetes components. This benchmark is, however, not applicable to Amazon EKS on AWS Fargate as you do not own or manage Kubernetes component configurations when using AWS Fargate.

Since the control plane is AWS' responsibility and we run no nodes, I guess there's no value in running the benchmark for us and we don't need to do this story...?

[mogul]

Here's a nice blog post on how to run the CISbenchmark from within the cluster; it provides a ready-to-go manifest and instructions for getting the results.

[mogul]

kube-bench doesn't even run in Fargate, at least without further tweaking:

  status:
    conditions:
    - lastProbeTime: null
      lastTransitionTime: "2021-04-28T00:28:26Z"
      message: 'Pod not supported on Fargate: fields not supported: HostPID, volumes
        not supported: var-lib-kubelet is of an unsupported volume Type, etc-systemd
        is of an unsupported volume Type, etc-kubernetes is of an unsupported volume
        Type'
      reason: POD_UNSUPPORTED_ON_FARGATE
      status: "False"
      type: PodScheduled
    phase: Pending
    qosClass: BestEffort

I'll do some further digging, but I'm increasingly thinking that the CIS benchmark doesn't apply to us...

[mogul]

Confirmed, everything kube-bench checks under EKS is related to nodes. I'll remove mention of it from our SSP.

[mogul]

Moving this to Done rather than closing it, as a record that we did our due diligence to determine that the CIS benchmark does not apply to our configuration.

[mogul]

Reopening since we're now using Managed Node Groups instead of Fargate and we want to demonstrate that we are maintaining this baseline.

[mogul]

Here's an up to date walkthrough of running the CIS EKS benchmark, including how to ensure the results are propagated to AWS Security Hub.

[mogul]

I'm checking out starboard and it looks pretty great.

[mogul]

Excellent video demonstrating Starboard. I'm going to give this a try.

[mogul]

PR pending

[nickumia-reisys]

We've successfully run benchmarks by implementing @mogul's solution

• https://github.com/GSA/datagov-brokerpak-eks/pull/90

We have also extracted the EKS CIS Benchmark reports from starboard using both (1) the Stock Amazon AMI and (2) the GSA-Hardened AMI.