Example/niplets little outdated?

[wazerstar]

Have some problems and I think its because some of the request items are outdated?

Having troubles correcting timezone to my zone I tried adding +0100 nothing happend would still say 1 hour behind. https://www.timeanddate.com/time/zone/denmark/copenhagen

remote_addr The remote_addr has changed to to remote_ip, and that I got working.

But cannot figure out how to let it show username/http.auth.user.id from basicauth and also how would I pull the REAL IP information, the remote_ip in my caddyfile is the remote_ip is cloudflare, so I would have to use X-Real-IP OR http.request.header.CF-Connecting-IP, but seems not to be working?

This is what I have.

format transform `{request>remote_ip} - {request>http.auth.user.id} [{ts}] {request>http.request.header.CF-Connecting-IP} {request>method} {request>uri} {request>proto} {status} {size}` {
    time_format "02/Jan/2006 15:04:05 +0100"

This would be an example it would show
162.158.134.79 - - [08/May/2022 23:03:58 +0500] - GET /vx HTTP/2.0 200 13328

[francislavoie]

You're not supposed to use placeholders here, you use the JSON path within the log.

For example, request>method means "in the log, look inside request, and inside that look at method".

So for the user ID, use {request>user_id} and for the headers, {request>headers>CF-Connecting-IP} I think.

For the time, see https://pkg.go.dev/time#pkg-constants. Changing the time zone in the format string does not change the time zone of the time being written. I don't think we have a way to change the time zone right now, they'll always be written as UTC.

[wazerstar]

You're not supposed to use placeholders here, you use the JSON path within the log.

For example, request>method means "in the log, look inside request, and inside that look at method".

So for the user ID, use {request>user_id} and for the headers, {request>headers>CF-Connecting-IP} I think.

I already did try both suggestions and came up empty.

For the time, see https://pkg.go.dev/time#pkg-constants. Changing the time zone in the format string does not change the time zone of the time being written. I don't think we have a way to change the time zone right now, they'll always be written as UTC.

Aha - Interesting that this is not in yet.

You guys are always fast!

I'm on 2.5.1

[francislavoie]

Using UTC for logs is generally much more useful because it guarantees that they'll always be in order. If your time changes (daylight savings time etc) then suddenly logs will be written plus or minus one hour in the other direction, which can mess things up if you're using some tooling to ingest your logs.

[francislavoie]

For the other 2 headers, it should be {user_id} and {request>headers>CF-Connecting-IP}; see the structure here https://caddyserver.com/docs/logging#structured-logs (I wrote the above on my phone so I didn't double-check where they were placed)

[wazerstar]

For the other 2 headers, it should be {user_id} and {request>headers>CF-Connecting-IP}; see the structure here https://caddyserver.com/docs/logging#structured-logs (I wrote the above on my phone so I didn't double-check where they were placed)

You where correct about {user_id} I had request in front of that, that's why I never got it work in first place.

the cloudflare thing http.request.header.CF-Connecting-IP is not working

I tried {request>headers>CF-Connecting-IP}

But {request>headers>X-Forwarded-For} does work?

[francislavoie]

It depends what headers are actually in the request. If you log normally without format transform, you can see what's actually available to use.

[wazerstar]

It depends what headers are actually in the request. If you log normally without format transform, you can see what's actually available to use.

Okay, I cannot see any CF- part in the logs the normal/standard way.

[wazerstar]

I found them via doing {request>headers}, but that does not help me :D

User=MASK - X-Forwarded-For=["MASK"] - remote_ip=162.158.134.139 - [09/May/2022 00:37:01 +0000] GET ["https"] MASK MASK HTTP/2.0 200 7739460 - - - VLC/3.0.16 LibVLC/3.0.16 - {"X-Forwarded-Proto":["https"],"Accept":["/"],"Cf-Connecting-Ip":["MASK"],"Accept-Encoding":["gzip"],"Cf-Ipcountry":["DK"],"X-Forwarded-For":["MASK"],"Cf-Ray":["MASK"],"Range":["bytes=0-"],"Accept-Language":["en_US"],"Cdn-Loop":["cloudflare"],"Cf-Visitor":["{"scheme":"https"}"],"Authorization":[],"User-Agent":["VLC/3.0.16 LibVLC/3.0.16"]}

Currently this would be what I'm using

format transform User={user_id} - X-Forwarded-For={request>headers>X-Forwarded-For} - remote_ip={request>remote_ip} - [{ts}] {request>method} {request>headers>X-Forwarded-Proto} {request>host} {request>uri} {request>proto} {status} {size} - {request>headers>Referer>[0]} - {request>headers>User-Agent>[0]} - {request>headers} {
        time_format "02/Jan/2006 15:04:05 -0700"

[francislavoie]

I don't understand. What's the problem?

[wazerstar]

I don't understand. What's the problem?

I would like to sort them in specified order like I do with the rest :)

[mohammed90]

I don't understand. What's the problem?

I would like to sort them in specified order like I do with the rest :)

They can't be generically sorted because the headers field is a map, and maps in Go are intentionally randomized to avoid relying on an assumption of maps being sorted.

[mohammed90]

I believe the confusion has been clarified. Let us know if you still have any questions or further clarifications are needed.