Caddy analyzation errors out with IPv4/6 is required

[Ducki]

I'm trying to analyze a log from Caddy. The Caddy output is a default log, with no customized output. The error of GoAccess (Version 1.4.6.) is:

[SETTING UP STORAGE access.log] {0} @ {0/s}  
==12172== GoAccess - Copyright (C) 2009-2020 by Gerardo Orellana  
==12172== https://goaccess.io - <hello@goaccess.io>  
==12172== Released under the MIT License.  
==12172==  
==12172== FILE: access.log  
==12172== Parsed 1 lines producing the following errors:  
==12172==  
==12172== IPv4/6 is required.  
==12172==  
==12172== Format Errors - Verify your log/date/time format

The raw access.log that Caddy generated is:

2021/03/12 08:57:48.541 info    http.log.access.log0    handled request {"request": {"remote_addr": "127.0.0.1:63746", "proto": "HTTP/1.1", "method": "GET", "host": "127.0.0.1", "uri": "/", "headers": {"Cache-Control": ["max-age=0"], "Connection": ["keep-alive"], "Upgrade-Insecure-Requests": ["1"], "If-Modified-Since": ["Thu, 11 Mar 2021 19:37:36 GMT"], "Accept-Language": ["de,en-US;q=0.7,en;q=0.3"], "Accept-Encoding": ["gzip, deflate"], "If-None-Match": ["\"qptl6o1eo\""], "User-Agent": ["Mozilla/5.0 (Macintosh; Intel Mac OS X 11.2; rv:86.0) Gecko/20100101 Firefox/86.0"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"]}}, "common_log": "127.0.0.1 - - [12/Mar/2021:09:57:48 +0100] \"GET / HTTP/1.1\" 304 0", "duration": 0.017638916, "size": 0, "status": 304, "resp_headers": {"Server": ["Caddy"], "Etag": ["\"qptl6o1eo\""]}}

Why does it tell me, that IPv4/6 is required?

Thanks in advance and best regards :)

[allinurl]

This was previously asked when reporting an issue with the JSON parser.

The line you posted above is not a valid JSON, however, I agree that goaccess should error that out instead of getting to the point where it attempts to parse it as non-JSON and thus displaying a confusing message.

Explicitly requesting the format json solved the problem!

log {
  output file /disk/site.com/log/access.log
  format json
}

Please let me know if that helps.

[Ducki]

Oh, I'm sorry – I was under the impression, that the default caddy format was also accepted. It works flawlessly with the "format json" option. Thank you very much – sorry for the kind of double question :)

[allinurl]

No problem at all and thank you for reporting it. I'll make sure the error message is corrected and pushed out in the upcoming release.

[samcro1967]

I am seeing the same issue. Not sure what I am missing.

First 15 lines in the log

{"level":"info","ts":1622829915.9538262,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
{"level":"info","ts":1622829915.9544854,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1622829915.9545262,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1622829915.9545336,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0003ca540"}
{"level":"info","ts":1622829915.9547102,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv1","http_port":80}
{"level":"debug","ts":1622829917.6321146,"logger":"http.authentication.providers.jwt","msg":"JWT access list entry","instance_name":"jwt-1","seq_id":0,"acl":{"action":"allow","values":["anonymous","guest","adm
in"],"claim":"roles"}}
{"level":"debug","ts":1622829917.6321757,"logger":"http.authentication.providers.jwt","msg":"JWT access list entry","instance_name":"jwt-1","seq_id":1,"acl":{"action":"allow","values":["superadmin"],"claim":"r
oles"}}
{"level":"debug","ts":1622829917.632212,"logger":"http.authentication.providers.jwt","msg":"JWT token configuration provisioned","instance_name":"jwt-1","trusted_tokens":[{"token_name":"access_token","token_li
fetime":900,"token_secret":"2sfhfhr6-8532-ahs6-kah6-pqst83jdjdbo"}],"auth_url_path":"https://auth.mydomain.net","token_sources":"header cookie query","token_types":"HS512","token_validator":{"TokenConfigs":[
{"token_name":"access_token","token_lifetime":900,"token_secret":"2sfhfhr6-8532-ahs6-kah6-pqst83jdjdbo"}],"AuthorizationHeaders":{"access_token":{},"jwt_access_token":{}},"Cookies":{"access_token":{},"jwt_acce
ss_token":{}},"QueryParameters":{"access_token":{},"jwt_access_token":{}},"Cache":{"Entries":{}},"AccessList":[{"action":"allow","values":["anonymous","guest","admin"],"claim":"roles"},{"action":"allow","value
s":["superadmin"],"claim":"roles"}],"TokenBackends":[{}],"TokenSources":["header","cookie","query"]},"token_validator_options":{"ValidateSourceAddress":false,"SourceAddress":"","ValidateBearerHeader":false,"Va
lidateMethodPath":false,"ValidateAccessListPathClaim":false,"ValidateAllowMatchAll":false,"Metadata":null,"Logger":null},"forbidden_path":""}

Global log option in my Caddyfile

    log {
        output file /data/access.log
                format json
        }

Output from cat access.log | sudo docker run --rm -i -e LANG=$LANG allinurl/goaccess -a -o html --log-format CADDY - > report.html

 [SETTING UP STORAGE -] {0} @ {0/s}
==1== GoAccess - Copyright (C) 2009-2020 by Gerardo Orellana
==1== https://goaccess.io - <hello@goaccess.io>
==1== Released under the MIT License.
==1==
==1== FILE: -
==1== Parsed 10 lines producing the following errors:
==1==
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1== IPv4/6 is required.
==1==
==1== Format Errors - Verify your log/date/time format

[allinurl]

@samcro1967 are you parsing the access log? I don't see the fields specified here in your sample log.

[samcro1967]

I had logs set to debug. Changed it to info and now it is working.