search

elastic / elasticsearch

Free and Open, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
68.54k stars 24.35k forks source link

Security Audit Log To Record Specific Properties of the Response #30039

Open elasticmachine opened 7 years ago

elasticmachine commented 7 years ago

Original comment by @skearns64:

There are some audit cases where users want the audit logs to contain properties from the response - specifically the number of hits, the number of documents returned, and took_time. This issue tracks adding this information as optional (off by default) to the audit log.

elasticmachine commented 7 years ago

Original comment by @uboness:

just to note here... today we don't have a way to correlate responses with their original requests. Not sure if it's required here, but if it is this will require additional thinking behind this feature.

elasticmachine commented 7 years ago

Original comment by @skearns64:

In this case, I don't think there is a requirement to link request with response, as long as we can log the User with the response details.

Eventually, it would be excellent to be able to link the request with the response. Perhaps by taskID? (do we have task ids for individual requests?)

elasticmachine commented 7 years ago

Original comment by @uboness:

not every request is associated with a task and multiple requests/responses can be associated with the same task. The one thing that binds a request to a response is the request ID which is generated on the transport layer, while the security filtering happens before that (at least that's how things work today... hence the need for additional thinking/work to create this association)

albertzaharovits commented 5 years ago

We had a brief discussion about this in our team meeting, and I've been tasked with sorting it out.

It is true that we still cannot correlate a request with a response - yet it is also true that this is not imperative in this case. Strictly speaking it's the user ( and maybe request_body) and response_body connection that is relevant. Moreover this is particular to search requests.

First of all, there is no unique id for a RestRequest, as there is for a Task, so a log entry of a response would not have a way to link to an id of a request. Therefore, either we add serial numbers to requests (and log them when we also log the request_body) or the new log entry should contain both the request and the paired response. Both alternatives require some work. The former would be interesting because Tasks are spawned from Requests and in a grander scheme of things we could fit tasks and requests in a session. The latter, more pragmatic to the issue at hand, should wrap the ActionListener<SearchResponse> in another listener which also packs the request and the log trail pipe. I reckon SecurityActionFilter#apply would be the place to hook in the wrapping listener. Also, the user should be available there to pack him together too. It sounds a little convoluted, but this is because security has been concerned with ingress (request filters...) and the whole plumbing is out reach by the time the response is ready to be sent.

But the user does stir up an interesting question: Suppose that we have a log line that contains the request_body and the response details we want, moreover X-Opaque-Id would also be there. Should this entry be an audit log entry? I does not look so to me. It could just as well be an entry in the elasticsearch.log. Now, if we throw in the requirement that responses have to be ascribed to the user, as a precursor to an accounting feature, then from this perspective, it is indeed more of an audit log entry.

@skearns64 @elastic/es-security I wonder, will the X-Opaque-Id, request_body and response parameters (hits, time, shards...) in a new log entry, be enough to meet this request? In other words can we keep the user out of it? In this case, the log entry can also be forwarded to the usual elasticsearch.log and the feature can sit in OSS.