allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
18.38k stars 1.11k forks source link

Token '' doesn't match specifier '%h' #1635

Closed almirus closed 4 years ago

almirus commented 4 years ago

if LOG have leading space, get error Token '' doesn't match specifier '%h' [SPACE]10.88.241.31 - - [24/Nov/2019:03:34:04 +0300] "GET /" 200 21139 "-" "-"

allinurl commented 1 year ago

@RyanZoou Please describe all those fields. Thanks.

RyanZoou commented 1 year ago

@allinurl Thank so much for your reply, the log fields config: log_format main '$time_local|$server_addr|$request|$status|$remote_user|$remote_addr|$http_user_agent|$http_referer|$host|$bytes_sent|$request_time|$upstream_response_time|$upstream_addr|$connection|$connection_requests|$uuid';

thoutamganesh66 commented 11 months ago

Can someone help me with the log format for my log file Here are the first few lines of my log file: "2023-11-07 00:00:08" hello.gb.com 201.144.130.113 GET "/mms/api/sk-applications/df819f8e" "" 200 138 "2023-11-07 00:00:11" hello.gb.com 201.144.130.113 GET "/index.html" "" 200 0 "2023-11-07 00:00:12" hello.gb.com 201.144.130.113 GET "/mms/api/get-current-time" "" 200 1 where the last number(138) is the response time taken by the API

allinurl commented 11 months ago

@thoutamganesh66 This should do it:

goaccess access.log --log-format='"%x" %v %h %m "%U" "%R" %s %b' --datetime-format='%Y-%m-%d %H:%M:%S'

I'm unsure about the purpose of the empty field before the status, marked as "". I included it as the referrer.

63070016 commented 10 months ago

Hi I got same problem with error.log I want to goaccess to error.log for nginx

MrWubbaLubbadubdub commented 8 months ago

Hello all, I'm also facing the similar issue. Requesting all to take a look at my access log and let me know, the correct command to run it.

====Error==== Token '0.248.48.40 - - [10/Dec/2021:15:59:05 +0530' doesn't match specifier '%h'

====Access Logs==== 11.11.11.11 - - [10/Dec/2021:15:59:07 +0530] "GET /scripts/formmailto.html HTTP/1.0" 403 199 11.11.11.11 - - [10/Dec/2021:15:59:07 +0530] "GET /scripts/mailtoform.html HTTP/1.0" 403 199 14.14.14.14 - - [10/Dec/2021:15:59:07 +0530] "GET /scripts/form.html HTTP/1.0" 403 199 12.12.12.12 - - [10/Dec/2021:15:59:07 +0530] "GET /formmail/formmail.html HTTP/1.0" 403 199 13.13.13.13 - - [10/Dec/2021:15:59:07 +0530] "GET /formmail/mailform.html HTTP/1.0" 403 199

A quick help would be appreciated @allinurl Thanks in advance.

allinurl commented 8 months ago

@MrWubbaLubbadubdub It appears that you're using the common log format. Please try the following and let me know of the outcome.

# goaccess access.log --log-format=COMMON
william-0129 commented 6 months ago

@allinurl Hello~ I tried using the official explanation as well as the nginx specific time code I still get the error

My nginx error.log looks like this:

2024/04/03 03:57:15 [error] 15893#15893: *71520311 upstream timed out (110: Connection timed out) while connecting to upstream, client: 1.1.1.1, server: 0.0.0.0:456, upstream: "8.8.8.8:123", bytes from/to client:0/0, bytes from/to upstream:0/0

goaccess error.log --log-format=COMMON The execution results are as follows

==23681== Token '2024/04/03' doesn't match specifier '%h' ==23681== Format Errors - Verify your log/date/time format

I also tried goaccess error.log --log-format='"%x" %v %h %m "%U" "%R" %s %b' --datetime-format='%Y-%m-%d % H:%M:%S' The results are as follows ==23684== Token '024/04/03 03:50:05 [error] 15893#15893: *71516579 upstream timed out (110: Connection timed out) while connecting to upstream, client: 8.8.8.8, server: 0.0 .0.0:123, upstream:' doesn't match specifier '%x'

2 of 2024 is gone

Thank you so much

allinurl commented 6 months ago

@william-0129, please note that goaccess works best with access log data. However, you could try pulling some of the error log data like this:

# goaccess error.log --log-format='%d %t [%e] %^ %^ %U, client: %h, server: %v %^' --date-format=%Y/%m/%d --time-format=%T --http-method=no --http-protocol=no --ignore-panel=REQUESTS_STATIC

The only thing is, it might depend on how the actual error messages are structured/delimited in your logs. But give it a try and let me know how it goes!

william-0129 commented 6 months ago

@allinurl Thank you very much for your help. So far, I have successfully exported to --real-time-html. Today, I tried again and it worked. It is the same as the first time I successfully output with your method, and then converted to --real-time-html. Only some dashboards don't have values. 截圖 2024-04-10 下午1 45 56

截圖 2024-04-10 下午1 47 33

allinurl commented 6 months ago

@william-0129 Give the --ignore-panel option a try. For more information on how to use it, refer to the man page.

QiJunLiang commented 3 months ago

Hello all, I'm also facing the similar issue. Help!!!

I use goaccess.exe E:/20240712-17/20240712.access.log -o E:/20240712-17/report12.html --log-format=COMBINED

====Error==== ==1983== GoAccess - version 1.9.1 - Apr 9 2024 11:37:46 ==1983== Config file: No config file used ==1983== https://goaccess.io - hello@goaccess.io ==1983== Released under the MIT License. ==1983== ==1983== FILE: E:/20240712-17/20240712.access.log ==1983== Parsed 10 lines producing the following errors: ==1983== ==1983== Token '169.62.88.198:61264' doesn't match specifier '%h' ==1983== Token '40.77.167.255:12825' doesn't match specifier '%h' ==1983== Token '169.62.88.198:61270' doesn't match specifier '%h' ==1983== Token '169.62.88.198:61271' doesn't match specifier '%h' ==1983== Token '169.62.88.198:61292' doesn't match specifier '%h' ==1983== Token '18.225.57.100:29246' doesn't match specifier '%h' ==1983== Token '169.62.88.198:61309' doesn't match specifier '%h' ==1983== Token '169.62.88.198:61312' doesn't match specifier '%h' ==1983== Token '18.225.57.100:29246' doesn't match specifier '%h' ==1983== Token '23.22.35.162:48765' doesn't match specifier '%h' ==1983== ==1983== Format Errors - Verify your log/date/time format

====Access Logs==== 169.62.88.198:61264 - - [12/Jul/2024:00:00:00 +0800] "GET /movies/vid-8724977.html?del_cacche_key=1 HTTP/1.1" 404 11392 "-" "-" "-" 28.264 28.264 40.77.167.255:12825 - - [12/Jul/2024:00:00:00 +0800] "GET /movie/difficult-people-how-to-deal-with-them-vid-8757122.html HTTP/2.0" 200 6542 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/116.0.1938.76 Safari/537.36" "-" 0.077 0.077 169.62.88.198:61270 - - [12/Jul/2024:00:00:00 +0800] "GET /movies/vid-23167.html?del_cacche_key=1 HTTP/1.1" 404 11390 "-" "-" "-" 28.227 28.227 169.62.88.198:61271 - - [12/Jul/2024:00:00:00 +0800] "GET /movies/vid-9984157.html?del_cacche_key=1 HTTP/1.1" 404 11392 "-" "-" "-" 28.289 28.289 169.62.88.198:61292 - - [12/Jul/2024:00:00:02 +0800] "GET /movies/vid-8381752.html?del_cacche_key=1 HTTP/1.1" 404 11392 "-" "-" "-" 28.202 28.202 18.225.57.100:29246 - - [12/Jul/2024:00:00:02 +0800] "GET /free-movie?c=2087&cid=10752&p=1147&y=2016 HTTP/2.0" 200 5113 "https://yesflicks.com/free-movie?c=2087&cid=10752&p=1147&y=2016" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "-" 7.119 7.119 169.62.88.198:61309 - - [12/Jul/2024:00:00:03 +0800] "GET /movies/vid-13435557.html?del_cacche_key=1 HTTP/1.1" 404 11393 "-" "-" "-" 28.180 28.180 169.62.88.198:61312 - - [12/Jul/2024:00:00:03 +0800] "GET /movies/vid-489521.html?del_cacche_key=1 HTTP/1.1" 404 11391 "-" "-" "-" 28.248 28.248 18.225.57.100:29246 - - [12/Jul/2024:00:00:03 +0800] "GET /free-tv-show?cid=10759&p=23&pre=r&y=2022 HTTP/2.0" 200 4669 "https://yesflicks.com/free-tv-show?cid=10759&p=23&pre=r&y=2022" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "-" 7.050 7.050 23.22.35.162:48765 - - [12/Jul/2024:00:00:03 +0800] "GET /movies/valentino-s-ghost-vid-449776.html HTTP/1.1" 200 6647 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)" "-" 7.067 7.067 18.222.162.142:16676 - - [12/Jul/2024:00:00:03 +0800] "GET /free-tv?c=2707&cid=10751&p=328&y=2014 HTTP/2.0" 200 5084 "https://yesflicks.com/free-tv?c=2707&cid=10751&p=328&y=2014" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" "-" 7.037 7.037 169.62.88.198:61326 - - [12/Jul/2024:00:00:03 +0800] "GET /movies/vid-13435557.html?del_cacche_key=1 HTTP/1.1" 404 11393 "-" "-" "-" 28.229 28.229

allinurl commented 3 months ago

@QiJunLiang The following should do it:

# goaccess access.log --log-format='%h:%^[%x] "%r" %s %b "%R" "%u" %^' --datetime-format='%d/%b/%Y:%H:%M:%S %z' --tz=America/New_York --date-spec=min

2024-07-18-081731_653x332_scrot