search

aquasecurity / kube-hunter

Hunt for security weaknesses in Kubernetes clusters
Apache License 2.0
4.66k stars 578 forks source link

K8s autodiscovery #453

Closed seeker89 closed 3 years ago

seeker89 commented 3 years ago

Description

This PR adds a new feature to kube-hunter to automatically list all nodes in a kubernetes cluster, and attempt hunting on all of their advertised addresses.

It adds two new flags:

This is handy in conjunction with --pod, when run from a pod (but it will require RBAC to list nodes).

All of the discovery logic for different clouds remains unaffected. This is useful when you run your own cluster, and the advertised IPs are routable.

Contribution Guidelines

Please Read through the Contribution Guidelines.

"BEFORE" and "AFTER" output

BEFORE

Feature not available.

AFTER

$ python -m kube_hunter --log INFO --k8s-auto-discover-nodes --kubeconfig ~/.kube/config 
2021-05-07 11:34:48,701 INFO kube_hunter.modules.report.collector Started hunting
2021-05-07 11:34:48,701 INFO kube_hunter.modules.report.collector Discovering Open Kubernetes Services
2021-05-07 11:34:48,701 INFO kube_hunter.conf.k8s Attempting to use kubeconfig file: /Users/mpawlikowsk1/.kube/config
2021-05-07 11:34:49,476 INFO kube_hunter.conf.k8s Listed 6 nodes in the cluster
2021-05-07 11:34:50,038 INFO kube_hunter.modules.report.collector Found open service "Kubelet API" at 10.11.14.54:10250
(...)
2021-05-07 11:34:50,830 INFO kube_hunter.modules.report.collector Found open service "API Server" at 10.11.15.34:6443
(...)

Nodes
+-------------+--------------+
| TYPE        | LOCATION     |
+-------------+--------------+
| Node/Master | 10.11.15.34  |
+-------------+--------------+
| Node/Master | 10.11.14.54  |
+-------------+--------------+
(...)

Detected Services
+-------------+--------------------+----------------------+
| SERVICE     | LOCATION           | DESCRIPTION          |
+-------------+--------------------+----------------------+
| Kubelet API | 10.11.15.34:10250  | The Kubelet is the   |
|             |                    | main component in    |
|             |                    | every Node, all pod  |
|             |                    | operations goes      |
|             |                    | through the kubelet  |
+-------------+--------------------+----------------------+
| Kubelet API | 10.11.14.54:10250  | The Kubelet is the   |
|             |                    | main component in    |
|             |                    | every Node, all pod  |
|             |                    | operations goes      |
|             |                    | through the kubelet  |
+-------------+--------------------+----------------------+
| API Server  | 10.11.15.34:6443   | The API server is in |
|             |                    | charge of all        |
|             |                    | operations on the    |
|             |                    | cluster.             |
+-------------+--------------------+----------------------+
(...)

Contribution checklist

Notes

This is an enhancement, so there was no active issue to reference. If you're happy to go forward with this, I'll add a test for the new function I introduced.

danielsagi commented 3 years ago

Hi @seeker89

Thats a really good addition, thanks! Ill take a deeper look at this in the next few days, and update you.

I think we will benefit alot from discovering nodes from the api server

seeker89 commented 3 years ago

Hi @seeker89

Thats a really good addition, thanks! Ill take a deeper look at this in the next few days, and update you.

I think we will benefit alot from discovering nodes from the api server

Thank you @danielsagi - looking forward to a review.

seeker89 commented 3 years ago

@danielsagi let me know if you'd like any more info!

seeker89 commented 3 years ago

Happy Monday! Can I help anyhow to make the review easier? @danielsagi Thanks in advance!

danielsagi commented 3 years ago

Hi @seeker89, Thanks for your patience. I did not have time to sit on this yet, I'm really sorry for the delay, Ill keep you posted this week!

seeker89 commented 3 years ago

Ping :) @danielsagi

danielsagi commented 3 years ago

Hi @seeker89 Really sorry for taking too long for this, But I'm with you now. Thank you for your patience! First of all I'll say, I love that you used the kubernetes package I think we should have done that a long time ago (of course until now we needed to do magic api stuff manually, but still).

I have 4 comments:

  1. Right now your implementation would not run with --pod because of this condition In order to run when pod scan is enabled, please add that inside FromPodHostDiscovery inside /kube_hunter/modules/discovery/hosts.py as well. I also would like for your feature to run automatically on PodScans!, (using the load_incluster_config) and so the check there should only be for the kubeconfig flag.

  2. because of the minor complexity of this action (mainly dependencies for the current env) I would not implement this under the conf module, but rather directly inside hosts.py. (I know it's a huge file, but in the future we plan to split it :) )

  3. I would add more explanation on the help of both arguments, including a reference on --k8s-auto-discover-nodes to the kubeconfig argument, and vice versa.

  4. Document this feature on the main README :) (under Scanning Options) provide a detailed explanation on when to use this. and the default behaviour if

  5. (optional) because I see us using the --kubeconfig flag in the future, You can add a new Event like UserKubeconfigEvent containing the config loaded object. This one is really optional

Again, I thank you for your time, hope you haven't lost motivation ;) this really is a wonderful feature. I'll give this one a boost.

danielsagi commented 3 years ago

On a second thought, Regarding my 2nd point. I think this deserves to be in a new kubernetes_client.py file under kube_hunter/modules/discovery. Wdyt?

Hi @seeker89 Really sorry for taking too long for this, But I'm with you now. Thank you for your patience! First of all I'll say, I love that you used the kubernetes package I think we should have done that a long time ago (of course until now we needed to do magic api stuff manually, but still).

I have 4 comments:

  1. Right now your implementation would not run with --pod because of this condition In order to run when pod scan is enabled, please add that inside FromPodHostDiscovery inside /kube_hunter/modules/discovery/hosts.py as well. I also would like for your feature to run automatically on PodScans!, (using the load_incluster_config) and so the check there should only be for the kubeconfig flag.
  2. because of the minor complexity of this action (mainly dependencies for the current env) I would not implement this under the conf module, but rather directly inside hosts.py. (I know it's a huge file, but in the future we plan to split it :) )
  3. I would add more explanation on the help of both arguments, including a reference on --k8s-auto-discover-nodes to the kubeconfig argument, and vice versa.
  4. Document this feature on the main README :) (under Scanning Options) provide a detailed explanation on when to use this. and the default behaviour if
  5. (optional) because I see us using the --kubeconfig flag in the future, You can add a new Event like UserKubeconfigEvent containing the config loaded object. This one is really optional

Again, I thank you for your time, hope you haven't lost motivation ;) this really is a wonderful feature. I'll give this one a boost.

seeker89 commented 3 years ago

On a second thought, Regarding my 2nd point. I think this deserves to be in a new kubernetes_client.py file under kube_hunter/modules/discovery. Wdyt?

Hi @seeker89 Really sorry for taking too long for this, But I'm with you now. Thank you for your patience! First of all I'll say, I love that you used the kubernetes package I think we should have done that a long time ago (of course until now we needed to do magic api stuff manually, but still). I have 4 comments:

  1. Right now your implementation would not run with --pod because of this condition In order to run when pod scan is enabled, please add that inside FromPodHostDiscovery inside /kube_hunter/modules/discovery/hosts.py as well. I also would like for your feature to run automatically on PodScans!, (using the load_incluster_config) and so the check there should only be for the kubeconfig flag.
  2. because of the minor complexity of this action (mainly dependencies for the current env) I would not implement this under the conf module, but rather directly inside hosts.py. (I know it's a huge file, but in the future we plan to split it :) )
  3. I would add more explanation on the help of both arguments, including a reference on --k8s-auto-discover-nodes to the kubeconfig argument, and vice versa.
  4. Document this feature on the main README :) (under Scanning Options) provide a detailed explanation on when to use this. and the default behaviour if
  5. (optional) because I see us using the --kubeconfig flag in the future, You can add a new Event like UserKubeconfigEvent containing the config loaded object. This one is really optional

Again, I thank you for your time, hope you haven't lost motivation ;) this really is a wonderful feature. I'll give this one a boost.

Hi @danielsagi I addressed points 1-4, on point 5, it would just be dead code for now, so I'd vote let's add it when it's needed. Let me know if this is good to go now :)

seeker89 commented 3 years ago

Happy Friday :)

danielsagi commented 3 years ago

@seeker89 Just ran my tests, everything looks great! Well probably publish a new release for this feature soon, so keep following :)