search

cruise-automation / k-rail

Kubernetes security tool for policy enforcement
Apache License 2.0
444 stars 55 forks source link

[Question] Is there a way to enforce only matching resources on a set of nodes? #85

Open js-timbirkett opened 3 years ago

js-timbirkett commented 3 years ago

OK, the question isn't the greatest 😬 - I'll try to explain a bit more:

We have a separate set of "build" nodes for CI/CD, but this could be applied to any scenario where you have a separate set of tainted nodes.

These nodes are typically short-lived and are used to allow docker-in-docker reducing the risk that a malicious app or user could run containers that potentially compromise or cause issues for other containers running on the host's Docker daemon.

We use nodeSelector, taints and tolerations to ensure that build agents run on build nodes and no other workloads get scheduled there.

It'd be nice if we could specifically deny (or allow) resources to run on these nodes with k-rail, and allow docker socket mounts on these nodes only based on label or taint. I'm not sure if this ability exists already or if it's a feature that others. would be interested in?

I can write a policy up for this and submit a PR?

js-timbirkett commented 3 years ago

I've thought about deploying OPA Gatekeeper and scribing up some rego, but I'd like to extend k-rail where it makes sense to do so and contribute back to the community that uses it :)

dustin-decker commented 3 years ago

This does not currently exist but would be a welcome feature addition.

Here are a few things to look at to enable it:

  1. A new exemptions field we need to be added and checked in: https://github.com/cruise-automation/k-rail/blob/master/policies/exemption.go#L26
  2. IsExempt would need to check if the given resource has any matching exemptions: https://github.com/cruise-automation/k-rail/blob/cafdc998eb624087d7b83f091fb241e456a64ea2/policies/exemption.go#L118
  3. You'll probably need to use the Pod decoder to obtain the nodeSelector, taints, or tolerations https://github.com/cruise-automation/k-rail/blob/master/resource/pod.go#L30
mark-adams commented 1 year ago

👋 The k-rail project has been deprecated and is no longer under active development. We recommend taking a look at OPA Gatekeeper to see if it might meet your needs going forward.

Thanks for your contribution(s) to the project!