search

zalando / skipper

An HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress
https://opensource.zalando.com/skipper/
Other
3.07k stars 347 forks source link

Customize access logs #551

Open evangelion1204 opened 6 years ago

evangelion1204 commented 6 years ago

Hi,

we are already very happy that the Flow-Id is part of the access logs. Currently I am investigating if we could also add additional data to the access logs, like the client id. Might be connected to #535 but we would need those headers permanently to be always able to trace errors.

Two option come to my mind here

Option 1

Add to context some sort of log context maybe quite similar to the request in the last comment of #423 This would allow filters or predicates to push data to the logger - in my case access logs. The pushed fields could then be simply appended at the end of the line.

Option 2

Have a custom field generator lambda function injected on access log handler generation or in the LogAccess directly that will build the fields on demand. With that solution data could be fetched from any field of the request like for the Flow-Id header but in a dynamical way and would also work with some small modifications with client-id.

Configuration

To make both version better configurable I would also assume that the log access format should be using named interpolation like the one I added to eskip/template. A log line could then instead of being super complicated:

    commonLogFormat = `%s - - [%s] "%s %s %s" %d %d`
    // format:
    // remote_host - - [date] "method uri protocol" status response_size "referer" "user_agent"
    combinedLogFormat = commonLogFormat + ` "%s" "%s"`
    // We add the duration in ms, a requested host and a flow id
    accessLogFormat = combinedLogFormat + " %d %s %s\n"

Be simply

accessLogFormat = `${host} - - [${timestamp}] "${method} ${uri} ${proto}" ...`

Only thing not yet solved would be the format of data types but I think we will find a solution here.

This format should be currently a default and possibly injected from outside to enable users to alter or append named placeholders that could be pushed via option 1 or option 2.

I would implement any of them but need to know which one you prefer. I assume it could even be a mix of both.

szuecs commented 4 years ago

Logs are now based on filter for example https://opensource.zalando.com/skipper/reference/filters/#enableaccesslog If you check the code maybe you can see already how this can be implemented.

AlexanderYastrebov commented 3 years ago

log access format should be using named interpolation like the one I added to eskip/template

This was unified across filters as https://github.com/zalando/skipper/blob/master/docs/reference/filters.md#template-placeholders and can be reused for access log formatting as well.

AlexanderYastrebov commented 3 years ago

Another thing not mentioned is whether access log pattern should be customized globally or per route or both.