google / cloud-forensics-utils

Python library to carry out DFIR analysis on the Cloud

Apache License 2.0

453 stars 89 forks source link

GKE end-to-end deployment quarantining functionality #380

Closed zkck closed 2 years ago

zkck commented 2 years ago

This draft PR includes end-to-end quarantining functionality for a Kubernetes Deployment in GKE.

Currently, the following steps are taken:

Workload-affected nodes are cordoned and abandoned from managed instance group
Workload pods are orphaned, via deleting the workload and its matching ReplicaSet
Workload nodes are isolated via deny-all GCP firewall rules

Still TODO:

[x] Options for isolation via a Kubernetes NetworkPolicy
[x] Options for draining workload-affected nodes from other pods
[x] Prompts guiding the user through the quarantining process options
[x] Tests (moved to separate issue)

codecov-commenter commented 2 years ago

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@1915355). Click here to learn what that means. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #380   +/-   ##
=======================================
  Coverage        ?   58.81%           
=======================================
  Files           ?       45           
  Lines           ?     3574           
  Branches        ?        0           
=======================================
  Hits            ?     2102           
  Misses          ?     1472           
  Partials        ?        0

Flag	Coverage Δ
nosetests	`58.81% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 1915355...35293d4. Read the comment docs.

zkck commented 2 years ago

Sample output for a cordon-orphan-firewall quarantining process.


/[REDACTED]/cloud-forensics-utils/venv/bin/python /[REDCATED]/cloud-forensics-utils/main.py
[...] [...] INFO     Cordoning Kubernetes node [REDACTED_NODE_1] holding [REDACTED_POD_1] pod from nginx deployment...
[...] [...] INFO     Abandoning instance [REDACTED_NODE_1] from cluster's managed instance group...
[...] [...] INFO     Cordoning Kubernetes node [REDACTED_NODE_2] holding [REDACTED_POD_2] pod from nginx deployment...
[...] [...] INFO     Abandoning instance [REDACTED_NODE_2] from cluster's managed instance group...
[...] [...] INFO     Cordoning Kubernetes node [REDACTED_NODE_3] holding [REDACTED_POD_3] pod from nginx deployment...
[...] [...] INFO     Abandoning instance [REDACTED_NODE_3] from cluster's managed instance group...
[...] [...] INFO     Orphaning Kubernetes workload nginx's pods...
[...] [...] INFO     Putting instance [REDACTED_NODE_1] into network quarantine...
[...] [...] INFO     ... (output from putting instance into network quarantine)
[...] [...] INFO     Putting instance [REDACTED_NODE_2] into network quarantine...
[...] [...] INFO     ... (output from putting instance into network quarantine)
[...] [...] INFO     Putting instance [REDACTED_NODE_3] into network quarantine...
[...] [...] INFO     ... (output from putting instance into network quarantine)```

giovannt0 commented 2 years ago

Side note: if the tests are being added to this PR, then it'd be better to request a review only once the PR is not draft anymore. If they're added separately, please mark this PR as ready for review