Draft RFC for adding saved object values to audit log

[jeramysoucy]

Describe the feature:

The concept of recording a "change set" in audit logs for saved object operations was raised in https://github.com/elastic/kibana/issues/177972).

Being able to log a change set would add overhead to each CRUD operation - we would need to retrieve the previous version of an object and perform a diff with the updated version. In addition, if concurrent writes are being executed on the same object we cannot be sure that the previous version retrieved is accurate (see optimistic concurrency). Pushing audit logging down to Elasticsearch might alleviate these issues, but Elasticsearch has zero context from which to create meaningful Kibana audit events.

We think a reasonable compromise could be to include the latest version, or subset thereof, of an object when an operation is audited. By tracing the audit logs, one would be able to generate the change set for each operation if needed. Due to the potentially large size of some saved objects, we thought of 3 ways to preventing runaway log file entry sizes:

• A per-object size cap: SO's that exceed this limit would be truncated. This would be a global setting in the audit logging system.
• SO type opt-in: the audit logger would only record the value of SO types that opt in. This would be settable during SO type registration.
• SO type field opt-in: the audit logger would only record a subset of SO fields for a type. This would be settable during SO type registration.

An RFC should be drafted to explore this idea and come to a consensus for the best approach to take in order to effectively support calculating SO change sets from an audit log.

October 2024 Update

This issue is marked as blocked pending a consensus on the specific requirements within the scope of Kibana's audit service. Product management will be assessing this, though it is not currently prioritized work.

In the meantime, to address customer enhancement requests, we have drafted an RFC for including saved object names in the audit log. Once the RFC is approved, this work will be assigned a priority and scheduled.

cc @bitzandeb

[elasticmachine]

Pinging @elastic/kibana-security (Team:Security)

[jeramysoucy]

The RFC has been drafted and is currently in internal team review.