Kubernetes Auditing/Logging

[eak13]

As an operator deploying or managing a site using Airshipctl, I want the ability to enable Kubernetes auditing so that I can diagnose and resolve problems, track deployment progress, verify security integrity, and provide a clear audit trail of K8s activities.

Treasuremap will provide a general template for enabling K8s auditing within the API server. This template should provide the following:

1. Provide a minimal audit policy of all user/API activity (log all requests at the metadata level)
2. The ability to retain logging/audit information & maintaining least privilege/modification prevention (e.g. logs should be read only, written to /var/log or some other read only path)
3. Provide a basic log retention options (e.g. 30 day retention)

As a reference, see the following from Airship 1 which provides metadata level auditing based on a user provided audit policy (lines 136 - 144). https://github.com/airshipit/promenade/blob/master/charts/apiserver/values.yaml

More on Kubernetes auditing can be found here: https://kubernetes.io/docs/tasks/debug-application-cluster/audit/

Acceptance Criteria

1. Verify minimal audit policy of all user/API activity (all requests logged at the metadata level)
2. Validate data captured (e.g. event type, date/time, user info, success/failure, etc.)
3. Verify log/audit information available to users & systems for analysis, auditing & troubleshooting purposes.
4. Verify log retention policy in place
5. Verify least privilege/modification prevention (e.g. logs are written to a read only directory)

[eak13]

For the deployment logging, there may be a tie in with https://github.com/airshipit/airshipctl/issues/335

[niharikabhavaraju]

please assign it to me

[eak13]

Please ensure that this is included in the airship-core type deployment

[michaelfix]

@niharikabhavaraju - Niharika, is there an updated status, and are we close(r) to finishing?

[niharikabhavaraju]

Status: Created a PS earlier https://review.opendev.org/c/airship/treasuremap/+/783456 (abandoned this because v2 branch no longer exists), created a new PS https://review.opendev.org/c/airship/treasuremap/+/791979 with fixes of review comments in older PS. Currently I'm testing the PS by deploying test-site in treasuremap, running into an error while deploying controlplane. I'm working on fixing the error.

[niharikabhavaraju]

https://review.opendev.org/c/airship/treasuremap/+/791979 moved to master

[pallavgupta]

Do we have any update on it?

[eak13]

@pallavgupta Once we're firmer on the logging requirements, I will update the issue accordingly. @niharikabhavaraju has done a fair amount of the development already, but we still need to nail down some of the parameter settings.

[vetbijaya]

I can take this one