search

allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
18.58k stars 1.11k forks source link

Token: doesn't match specifier '%s' #836

Closed thangamani-arun closed 7 years ago

thangamani-arun commented 7 years ago

1. Nginx Access Log :

- 90.74.144.22|-|[05/Jul/2017:10:01:01 +0800]|"GET /wp-content/plugins/wptouch-pro/themes/foundation/modules/wptouch-icons/font/wptouch-icons.woff?64777116 HTTP/1.1"|200|11400|"http://www.mysite.com/category/nation/2017/07/05/shell-standee-molest-whats-becoming-of-our-men/"|"Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_2 like Mac OS X) AppleWebKit/603.2.4 (KHTML, like Gecko) Mobile/14F89 [FBAN/FBIOS;FBAV/96.0.0.45.70;FBBV/60548545;FBDV/iPhone9,3;FBMD/iPhone;FBSN/iOS;FBSV/10.3.2;FBSS/2;FBCR/Ooredoo;FBID/phone;FBLC/en_GB;FBOP/5;FBRV/0]"|requesttime:0.000|respondtime:-|format:wptouch|cache:-|bot:0|.
- 115.132.71.17|-|[05/Jul/2017:10:01:01 +0800]|"GET /category/nation/2017/07/05/girl-8-caned-by-headmistress-until-she-bled-claims-parent/ HTTP/1.1"|200|21254|"http://www.mysite.com/"|"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36"|requesttime:0.000|respondtime:-|format:fullversion|cache:HIT|bot:0|.
- 113.210.185.243|-|[05/Jul/2017:10:01:01 +0800]|"GET /category/Bahasa/feed/ HTTP/1.1"|200|52905|"-"|"Dalvik/2.1.0 (Linux; U; Android 6.0.1; SM-N910C Build/MMB29K)"|requesttime:0.000|respondtime:-|format:fullversion|cache:HIT|bot:0|.
- 210.187.164.18|-|[05/Jul/2017:10:01:01 +0800]|"GET /category/nation/2017/07/04/ppbm-veep-wants-disciplinary-action-against-syed-saddiq/ HTTP/1.1"|200|18291|"-"|"Mozilla/5.0 (Linux; U; Android 4.0.4; en-gb; GT-N7000 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30"|requesttime:0.000|respondtime:-|format:wptouch|cache:HIT|bot:0|.
- 183.171.75.149|-|[05/Jul/2017:10:01:01 +0800]|"GET /category/nation/2017/06/20/isas-move-to-spad-a-promotion-says-pkrs-wong-chen/embed/ HTTP/1.1"|200|12287|"http://www.mysite.com/"|"Mozilla/5.0 (Linux; Android 6.0.1; Mi Note 2 Build/MXB48T; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/59.0.3071.125 Mobile Safari/537.36"|requesttime:0.249|respondtime:0.249|format:wptouch|cache:MISS|bot:0|.
- 104.239.198.35|-|[05/Jul/2017:10:01:01 +0800]|"GET /category/business/2017/07/05/philippines-annual-inflation-eases-for-2nd-straight-month-in-june/ HTTP/1.1"|200|19948|"-"|"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"|requesttime:0.000|respondtime:-|format:fullversion|cache:HIT|bot:0|.
- 183.171.75.149|-|[05/Jul/2017:10:01:01 +0800]|"GET /category/nation/2017/07/03/does-our-vote-mean-nothing-taxi-drivers-ask-bn/embed/ HTTP/1.1"|200|12273|"http://www.mysite.com/"|"Mozilla/5.0 (Linux; Android 6.0.1; Mi Note 2 Build/MXB48T; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/59.0.3071.125 Mobile Safari/537.36"|requesttime:0.273|respondtime:0.273|format:wptouch|cache:MISS|bot:0|.
- 211.25.127.218|-|[05/Jul/2017:10:01:02 +0800]|"GET /category/opinion/2017/07/05/raising-issues-on-extremism-not-the-same-as-condoning-them/ HTTP/1.1"|200|21022|"http://www.mysite.com/category/nation/2017/07/05/shell-standee-molest-whats-becoming-of-our-men/"|"Mozilla/5.0 (Windows NT 6.1; rv:51.0) Gecko/20100101 Firefox/51.0"|requesttime:0.000|respondtime:-|format:fullversion|cache:HIT|bot:0|.
- 211.24.122.7|-|[05/Jul/2017:10:01:02 +0800]|"POST /wp-admin/admin-ajax.php HTTP/1.1"|200|114|"http://www.mysite.com/wp-admin/post-new.php"|"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"|requesttime:0.159|respondtime:0.159|format:fullversion|cache:-|bot:0|.
- 183.171.69.70|-|[05/Jul/2017:10:01:02 +0800]|"GET /category/Bahasa/feed/ HTTP/1.1"|200|52905|"-"|"Dalvik/2.1.0 (Linux; U; Android 6.0.1; SM-G532G Build/MMB29T)"|requesttime:0.000|respondtime:-|format:fullversion|cache:HIT|bot:0|.

2. Goaccess Conf: 

cat /etc/goaccess.conf 
time-format %T
date-format %d/%b/%Y
log_format %h|-|[%d:%t %^]|"%r"|%s|%b|"%R"|"%u"|requesttime:%T|respondtime:%^|format:%^|cache:%^|bot:%^|%^

Parse Error:

./goaccess-1.2/goaccess access.log -o report.html -p /etc/goaccess.conf 
access.log
Parsed 10 lines producing the following errors:

Token 'GET /wp-content/plugins/wptouch-pro/themes/foundation/modules/wptouch-icons/font/wptouch-icons.woff?64777116 HTTP/1.1"' doesn't match specifier '%s'
Token 'GET /category/nation/2017/07/05/girl-8-caned-by-headmistress-until-she-bled-claims-parent/ HTTP/1.1"' doesn't match specifier '%s'
Token 'GET /category/Bahasa/feed/ HTTP/1.1"' doesn't match specifier '%s'
Token 'GET /category/nation/2017/07/04/ppbm-veep-wants-disciplinary-action-against-syed-saddiq/ HTTP/1.1"' doesn't match specifier '%s'
Token 'GET /category/nation/2017/06/20/isas-move-to-spad-a-promotion-says-pkrs-wong-chen/embed/ HTTP/1.1"' doesn't match specifier '%s'
Token 'GET /category/business/2017/07/05/philippines-annual-inflation-eases-for-2nd-straight-month-in-june/ HTTP/1.1"' doesn't match specifier '%s'
Token 'GET /category/nation/2017/07/03/does-our-vote-mean-nothing-taxi-drivers-ask-bn/embed/ HTTP/1.1"' doesn't match specifier '%s'
Token 'GET /category/opinion/2017/07/05/raising-issues-on-extremism-not-the-same-as-condoning-them/ HTTP/1.1"' doesn't match specifier '%s'
Token 'POST /wp-admin/admin-ajax.php HTTP/1.1"' doesn't match specifier '%s'
Token 'GET /category/Bahasa/feed/ HTTP/1.1"' doesn't match specifier '%s'

Format Errors - Verify your log/date/time format

why am I getting parsing error, even-though I use proper format ?

Kindly give correct logformat to generate report without any parse error.

allinurl commented 7 years ago 
goaccess access.log --log-format='%^ %h|%^[%d:%t %^"%m %U %H"%^|%s|%b|"%R"|"%u"%^:%T%^|%^' --date-format=%d/%b/%Y --time-format=%T

or 

log-format %^ %h|%^[%d:%t %^"%m %U %H"%^|%s|%b|"%R"|"%u"%^:%T%^|%^
time-format %T
date-format %d/%b/%Y
allinurl commented 7 years ago

Closing this. Feel free to reopen it as needed.

ptmjyothish commented 6 years ago

Hi Allinuri,

I'm pretty new to this concept. Please help me to generate goaccess log report for the following format:

42.107.223.118 - [03/May/2018:13:07:38 +0530] "GET /favicon.ico HTTP/1.1" 404
64.233.173.143 - [03/May/2018:13:07:38 +0530] "GET /favicon.ico HTTP/1.1" 404
117.243.27.62 - [03/May/2018:13:07:39 +0530] "GET / HTTP/1.1" 200
168.235.205.79 - [03/May/2018:13:07:39 +0530] "POST /schoolcode_slc.php HTTP/1.1" 200

Also, I will share full log if needed.

allinurl commented 6 years ago

Please try this:

goaccess access.log --log-format='%h %^[%d:%t %^] "%r" %s' --date-format=%d/%b/%Y --time-format=%T --ignore-panel=OS --ignore-panel=BROWSERS --ignore-panel=REFERRING_SITES
ptmjyothish commented 6 years ago

Thank you very much....I will try and let you know...03.05.2018, 23:33, "Gerardo O." notifications@github.com:Please try this: goaccess access.log --log-format='%h %^[%d:%t %^] "%r" %s' --date-format=%d/%b/%Y --time-format=%T --ignore-panel=OS --ignore-panel=BROWSERS --ignore-panel=REFERRING_SITES

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.

ptmjyothish commented 6 years ago

Hi, allinurl Please find the attached sample squid log file details for your reference. I've tried the following code but showing error: goaccess access.log --log-format='%x.%^ %~ %L %h %^/%s %b %m %U %^' --date-format=%s --time-format=%s --http-protocol=no --ignore-panel=BROWSERS --ignore-panel=OS --ignore-panel=REFERRING_SITES ===========================================================1527030124.110     48 10.64.15.55 TCP_DENIED/403 4157 GET http://master2.teamviewer.com/din.aspx? - HIER_NONE/- text/html1527030126.057    817 10.64.10.153 TCP_TUNNEL/200 792 CONNECT lc92.dsr.livefyre.com:443 - HIER_DIRECT/54.159.86.114 -1527030128.837    212 10.79.16.229 TCP_MISS/200 1739 GET http://en-gb.appex-rf.msn.com/cgtile/v1/en-GB/HealthAndFitness/Home.xml - HIER_DIRECT/23.57.213.123 application/xml1527030128.838    214 10.79.16.229 TCP_MISS/200 1760 GET http://en-gb.appex-rf.msn.com/cgtile/v1/en-GB/News/Today.xml - HIER_DIRECT/23.57.213.123 application/xml1527030128.894     55 10.79.16.229 TCP_MISS/200 1754 GET http://en-gb.appex-rf.msn.com/cgtile/v1/en-GB/Sports/Today.xml - HIER_DIRECT/23.57.213.123 application/xml1527030128.961    336 10.79.16.229 TCP_MISS/200 1802 GET http://finance.services.appex.bing.com/Market.svc/AppTileV2? - HIER_DIRECT/104.122.113.187 application/xml1527030129.932  60980 10.79.6.127 TCP_MISS/200 377 GET http://su.ff.avast.com/R/A3gKIDhhMzZhMWJiOWE5MjRlZDdhNGMwOWQ4ZDY4MTAzYzYzEgQCEwIYGHgiAQAqBwgEEKb9k2AqBwgDEMrU7F4yCggAEJz_k2AYgAI45JeEkAFCIGYc9Nid2GvX4vdNTULCYoElfhkv76LLaD9LjPcahIHPSICDmBg= - HIER_DIRECT/5.45.62.53 application/octet-stream1527030130.389   9106 10.79.23.181 TCP_TUNNEL/200 1812 CONNECT play.google.com:443 - HIER_DIRECT/172.217.26.174 -1527030131.850      0 10.83.45.1 TCP_DENIED/403 4049 CONNECT ping3.teamviewer.com:443 - HIER_NONE/- text/html1527030132.115      0 10.64.15.55 TCP_DENIED/403 4050 CONNECT ping3.teamviewer.com:443 - HIER_NONE/- text/html1527030132.118      0 10.64.15.55 TCP_DENIED/403 4151 GET http://ping3.teamviewer.com/din.aspx? - HIER_NONE/- text/html1527030132.170     49 10.64.15.55 TCP_DENIED/403 4160 GET http://master14.teamviewer.com/din.aspx? - HIER_NONE/- text/html1527030133.435 3600236 10.64.255.74 TCP_TUNNEL/200 93653 CONNECT 0.docs.google.com:443 - HIER_DIRECT/74.125.24.189 -1527030135.906      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html1527030135.908      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html =========================================================================== 03.05.2018, 23:33, "Gerardo O." notifications@github.com:Please try this: goaccess access.log --log-format='%h %^[%d:%t %^] "%r" %s' --date-format=%d/%b/%Y --time-format=%T --ignore-panel=OS --ignore-panel=BROWSERS --ignore-panel=REFERRING_SITES

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.

allinurl commented 6 years ago

@ptmjyothish Please attach the log to a new post since right now it's unreadable. Thanks.

ptmjyothish commented 6 years ago

Sorry for the inconvenience caused.

Please find the snippet. Also attached sample file for your reference.

1527045669.352     46 10.108.139.1 TCP_MISS/200 715 POST http://10.5.9.175/sync3/Service.asmx - HIER_DIRECT/10.5.9.175 text/xml
1527045669.354      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html
1527045669.361      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html
1527045669.389      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html
1527045669.399     30 10.108.139.1 TCP_MISS/200 2008 POST http://10.5.9.175/sync3/Service.asmx - HIER_DIRECT/10.5.9.175 text/xml
1527045669.413      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html
1527045669.414      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html
1527045669.429      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html
1527045669.453      0 10.76.20.2 TCP_DENIED/403 4116 CONNECT af.opera.com:443 - HIER_NONE/- text/html

access.log

ptmjyothish commented 6 years ago

1527045669.399     30 10.108.139.1 TCP_MISS/200 2008 POST http://10.5.9.175/sync3/Service.asmx - HIER_DIRECT/10.5.9.175 text/xml23.05.2018, 20:13, "Gerardo O." notifications@github.com:@ptmjyothish Please attach the log to a new post since right now it's unreadable. Thanks.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

allinurl commented 6 years ago

@ptmjyothish This is actually the same issue as #988. I need to look into this and I'll post back as soon as I have a fix. Thanks for reporting this.

ptmjyothish commented 6 years ago

 Dear Allinurl, Kindly help me to find out the DNS log format to parse with go access. =================================================

23-Jul-2018 12:58:52.604 client 74.125.190.24#35804 (www.keralapoliceacademy.gov.in): view external: query: www.keralapoliceacademy.gov.in IN A -EDC (10.1.2.2)  23-Jul-2018 12:58:52.665 client 218.248.112.98#30303 (HUB3.MymEsS.nEt.anCL02GP.IkM.iN): view external: query: HUB3.MymEsS.nEt.anCL02GP.IkM.iN IN A -EDC (10.1.2.2)  23-Jul-2018 12:58:52.752 client 218.248.112.98#62739 (Hub3.MYmEsS.nET.Ikm.in): view external: query: Hub3.MYmEsS.nET.Ikm.in IN A -EDC (10.1.2.2)

=================================================24.05.2018, 17:30, "Gerardo O." notifications@github.com:@ptmjyothish This is actually the same issue as #988. I need to look into this and I'll post back as soon as I have a fix. Thanks for reporting this.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

allinurl commented 6 years ago

@ptmjyothish Try this:

goaccess log --log-format='%d %t.%^ %^ %h#%^ (%v) %^ %^ %^ %R %^ %U %^' --date-format=%d-%b-%Y --time-format=%T --ignore-panel=OS --ignore-panel=BROWSERS --ignore-panel=REFERRING_SITES --http-protocol=no --http-method=no
petqa commented 6 years ago

Hello. Trying a lot of log format. Can you help with this?

2018/07/23 10:57:20 [crit] 9732#9732: *1505159 open() "/var/www/url.ru/web/upload/resize_cache/iblock/6cc/320_174_0619711fa078991f0a23d032687646b21/6ccd9c8abf2806a9d52cd4cff73a4a5f.jpg" failed (24: Too many open files), client: 104.223.0.52, server: url.ru, request: "GET /upload/resize_cache/iblock/6cc/320_174_0619711fa078991f0a23d032687646b21/6ccd9c8abf2806a9d52cd4cff73a4a5f.jpg HTTP/1.1", host: "www.url.ru", referrer: "https://www.url.ru/"
2018/07/23 10:57:20 [crit] 9732#9732: *1497039 open() "/var/www/url.ru/web/sendsay_push_sw.js.gz" failed (24: Too many open files), client: 80.194.189.94, server: url.ru, request: "GET /sendsay_push_sw.js HTTP/2.0", host: "www.url.ru", referrer: "https://www.url.ru/sendsay_push_sw.js"
2018/07/23 10:57:21 [crit] 9732#9732: *1505425 open() "/var/www/url.ru/web/upload/resize_cache/iblock/b55/270_350_1/b55a0969760ccaeb8687a7fc98a0169c.jpg" failed (24: Too many open files), client: 185.236.117.223, server: url.ru, request: "GET /upload/resize_cache/iblock/b55/270_350_1/b55a0969760ccaeb8687a7fc98a0169c.jpg HTTP/1.1", host: "www.url.ru", referrer: "https://www.url.ru/saloni-optiki/zapis/?CLINIC_ID=7971"
2018/07/23 10:57:22 [crit] 9732#9732: *1505159 open() "/var/www/url.ru/web/upload/resize_cache/iblock/887/1170_460_1619711fa078991f0a23d032687646b21/88719cb47d1b8e3d1e0cd44c6b0efc1f.jpg" failed (24: Too many open files), client: 104.223.0.52, server: url.ru, request: "GET /upload/resize_cache/iblock/887/1170_460_1619711fa078991f0a23d032687646b21/88719cb47d1b8e3d1e0cd44c6b0efc1f.jpg HTTP/1.1", host: "www.url.ru", referrer: "https://www.url.ru/"
2018/07/23 10:57:22 [crit] 9732#9732: accept4() failed (24: Too many open files)
2018/07/23 10:57:24 [crit] 9732#9732: accept4() failed (24: Too many open files)
2018/07/23 10:57:24 [crit] 9732#9732: *1480209 open() "/var/www/url.ru/web/upload/iblock/713/713001e847571a164768f760c232f8d2.jpg" failed (24: Too many open files), client: 184.176.6.131, server: url.ru, request: "GET /upload/iblock/713/713001e847571a164768f760c232f8d2.jpg HTTP/2.0", host: "www.url.ru", referrer: "https://www.url.ru/doctor/8214/"
2018/07/23 10:57:24 [alert] 9732#9732: *1480209 socket() failed (24: Too many open files) while connecting to upstream, client: 184.176.6.131, server: url.ru, request: "GET /upload/iblock/713/713001e847571a164768f760c232f8d2.jpg HTTP/2.0", upstream: "fastcgi://127.0.0.1:9010", host: "www.url.ru", referrer: "https://www.url.ru/doctor/8214/"
2018/07/23 10:57:25 [crit] 9732#9732: *1494377 open() "/var/www/url.ru/web/upload/resize_cache/iblock/887/1170_460_1619711fa078991f0a23d032687646b21/88719cb47d1b8e3d1e0cd44c6b0efc1f.jpg" failed (24: Too many open files), client: 63.38.178.22, server: url.ru, request: "GET /upload/resize_cache/iblock/887/1170_460_1619711fa078991f0a23d032687646b21/88719cb47d1b8e3d1e0cd44c6b0efc1f.jpg HTTP/2.0", host: "www.url.ru", referrer: "https://www.url.ru/"
2018/07/23 10:57:25 [crit] 9732#9732: *1494377 open() "/var/www/url.ru/web/upload/resize_cache/iblock/da0/1170_460_1619711fa078991f0a23d032687646b21/da05ab60c10f411a73f3558a2f06f304.jpg" failed (24: Too many open files), client: 63.38.178.22, server: url.ru, request: "GET /upload/resize_cache/iblock/da0/1170_460_1619711fa078991f0a23d032687646b21/da05ab60c10f411a73f3558a2f06f304.jpg HTTP/2.0", host: "www.url.ru", referrer: "https://www.url.ru/"
allinurl commented 6 years ago

@petqa Please try:

goaccess access.log --log-format='%d %t %^ %^ %^ "%^" %^ (%^) %^ %^ %h, %^ %v, %^ "%r", %^ %^ %^ "%R"' --date-format=%Y/%m/%d --time-format=%T --ignore-panel=OS --ignore-panel=BROWSERS
petqa commented 6 years ago

thank you very much, it's worked! can you also help with access.log? thanks.

203.99.185.124 - - [23/Jul/2018:10:53:59 +0300] "GET /local/ajax/ajax_record.php?action=subwayDropdown&CITY_ID=212 HTTP/2.0" 500 588 "https://www.lensmaster.ru/saloni-optiki/?actionpay=22d38939-66d3-6f8f-d534-0164c620fe7d.182535&utm_source=actionpay&utm_medium=cpa&utm_campaign=cpa_actionpay&utm_term=182535" "Mozilla/5.0 (Linux; Android 6.0.1; SM-G532F Build/MMB29T; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/56.0.2924.87 Mobile Safari/537.36"
203.99.185.124 - - [23/Jul/2018:10:53:59 +0300] "GET /local/ajax/ajax_record.php?action=updateMap&CITY_ID=212&SUBWAY_ID= HTTP/2.0" 500 588 "https://www.lensmaster.ru/saloni-optiki/?actionpay=22d38939-66d3-6f8f-d534-0164c620fe7d.182535&utm_source=actionpay&utm_medium=cpa&utm_campaign=cpa_actionpay&utm_term=182535" "Mozilla/5.0 (Linux; Android 6.0.1; SM-G532F Build/MMB29T; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/56.0.2924.87 Mobile Safari/537.36"
114.124.141.64 - - [23/Jul/2018:10:53:59 +0300] "GET /local/ajax/ajax_record.php?action=subwayDropdown&CITY_ID=212 HTTP/1.1" 502 568 "https://www.lensmaster.ru/saloni-optiki/?actionpay=2cd49512-6ad3-6f8f-d534-0164c620f558.182535&utm_source=actionpay&utm_medium=cpa&utm_campaign=cpa_actionpay&utm_term=182535" "Mozilla/5.0 (Linux; Android 4.4.2; EVERCOSS A7R Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36"
119.160.102.182 - - [23/Jul/2018:10:53:59 +0300] "GET /saloni-optiki/?actionpay=b86d75b3-5fd3-6f8f-d534-0164c6212360.182535&utm_source=actionpay&utm_medium=cpa&utm_campaign=cpa_actionpay&utm_term=182535 HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Linux; Android 6.0; QMobile X700 PRO Build/MRA58K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/48.0.2564.106 Mobile Safari/537.36"
68.45.67.181 - - [23/Jul/2018:10:53:59 +0300] "GET /saloni-optiki/?actionpay=29fb3b40-8dd3-6f8f-d534-0164c621328a.182535&utm_source=actionpay&utm_medium=cpa&utm_campaign=cpa_actionpay&utm_term=182535 HTTP/1.1" 200 21464 "-" "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36"
allinurl commented 6 years ago

@petqa 

goaccess access.log --log-format=COMBINED
BBF0047 commented 6 years ago

1.Ningx Log Format

log_format main "$clientRealIp | $remote_addr | $time_local | $remote_user | $request_method | $request |"
                    "$status | $body_bytes_sent | $content_length | $http_referer | $http_user_agent |"
                    "$upstream_addr | $upstream_response_time | $request_time";
  1. Nginx Access Log : 
    222.210.136.110 | 157.119.232.7 | 09/Oct/2018:00:05:00 +0800 | - | GET | GET /Test?callback=localJsonpCallback&url=http://cc.nhnxs.com/%E7%BA%BF%E8%B7%AF-II&sendDate=1539014699979&_=1539014699943 HTTP/1.1 |200 | 87 | - | http://www.jxgy9.com/ | Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1 |103.6.84.135:80 | 0.012 | 0.012

    3.Goaccess Conf:

    time-format %T
date-format %d/%b/%Y
log_format %^ %h|%h|%d:%t %^ %m %U %H %^|%s|%b|%R|%u|%^|%T|%^

    4.Parse Error:

    
[root@ins-6t6376r2 access]# goaccess web_cdn.access.log -o /root/report.html --log-format=COMBINED
web_cdn.access.logs]
Parsed 10 lines producing the following errors:

Token '157.119.232.11 | 09/Oct/2018' doesn't match specifier '%d' Token '42.81.144.77 | 09/Oct/2018' doesn't match specifier '%d' Token '157.119.232.11 | 09/Oct/2018' doesn't match specifier '%d' Token '114.112.160.30 | 09/Oct/2018' doesn't match specifier '%d' Token '113.142.88.31 | 09/Oct/2018' doesn't match specifier '%d' Token '157.119.232.4 | 09/Oct/2018' doesn't match specifier '%d' Token '157.119.232.7 | 09/Oct/2018' doesn't match specifier '%d' Token '113.107.151.52 | 09/Oct/2018' doesn't match specifier '%d' Token '157.119.232.5 | 09/Oct/2018' doesn't match specifier '%d' Token '42.81.144.78 | 09/Oct/2018' doesn't match specifier '%d'

Format Errors - Verify your log/date/time format


why am I getting parsing error, even-though I use proper format ?

Kindly give correct logformat to generate report without any parse error.
allinurl commented 6 years ago

@BBF0047 Please open a new issue, it looks like I'd have to add full support when the log is delimited by pipes.

905839698 commented 5 years ago 
120.27.173.216 - - [21/Dec/2018:14:25:40 +0800] "GET http://www.aiyilearning.com/course_set/6778/manage/base" "200" 11911 "http://www.aiyilearning.com/course_set/6778/manage/base" "127.0.0.1:9000" "0.052" "0.052" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 SE 2.X MetaSr 1.0" "14.104.200.236"
allinurl commented 5 years ago

@905839698 

goaccess access.log --log-format='%h %^[%d:%t %^] "%m %U" "%s" %b "%R" "%v" "%T" %^ "%u" %^' --date-format=%d/%b/%Y --time-format=%T --http-protocol=no
elliotliu commented 5 years ago

Hello. Trying a lot of log format. Can you help with this?

58.210.27.10 - - [20/Aug/2019:00:00:00 +0800] "GET /perplat/css/public/images/printBtn.gif HTTP/1.1" 200 2032 "https://ebank.suzhoubank.com/perplat/css/public/public.css" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"

goaccess -f access_log.log --log-format='%h %^[%d:%t %^] "%m %U" "%s" %b "%R" "%v" "%T" %^ "%u" %^' --date-format=%d/%b/%Y --time-format=%T --ignore-panel=OS --ignore-panel=BROWSERS --http-method=no --http-protocol=no

Token '20/Aug/2019' doesn't match specifier '%d'

allinurl commented 5 years ago

@elliotliu Please try the following using the latest version:

goaccess access_log.log --log-format=COMBINED
fecori commented 5 years ago

Hi, i have this log:

"10.20.4.158" - - [12/Sep/2019:03:44:23 -0500] "-" "GET /impresa/larepublica-lima/09-09-2019 HTTP/1.1" 200 17595 "-" "-" "173"

@allinurl how to use goaccess log format?

and have this error: IPv4/6 is required.

thnx!

allinurl commented 5 years ago

@fecori please try:

goaccess access.log --log-format='"%h" %^[%d:%t %^] "%^" "%r" %s %b "%R" "%u" "%D"' --date-format=%d/%b/%Y --time-format=%T
fecori commented 5 years ago

@fecori please try:

goaccess access.log --log-format='"%h" %^[%d:%t %^] "%^" "%r" %s %b "%R" "%u" "%D"' --date-format=%d/%b/%Y --time-format=%T

thnx!!!!

FeZoli commented 4 years ago

Hi Guys! Please note that your current locale setting has a serious impact on the evaluation of dates. Always check with 'date' command before using goaccess, like this: $ date + %d/%b/%Y

In my case it returned '16/Ápr/2020', which would not mach anywhere in the log!

After setting LC_ALL=en_US.UTF-8, evrything works as expected.

exitmsconfig commented 4 years ago

84.236.19.203 - - [07/Aug/2020:03:46:55 +0800] "GET / HTTP/1.1" czguochuang.com 200 19265 "https://brandnewblogs.com/?domain=czguochuang.com" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-" 47.52.201.122:443 200 - "text/html; charset=utf-8" 0.256 > 0.259

How can I configure my time,

I execute the command: goaccess log_jx_https_access.log --log-format='%d %t %^ %^ %^ "%^" %^ (%^) %^ %^ %h, %^ %v, %^ " %r", %^ %^ %^ "%R"'

Error message: Token '112.206.121.228' doesn't match specifier'%d'

date-format %Y/%m/%d time-format %H:%M:%S log-format %d %t %^: %^: %^: %h,%^"%r",%^"%^",%^"%v",%^"%R"

How can I configure it to be normal

I execute to view the system time format date -R

Results: Fri, 07 Aug 2020 03:51:30 +0800

Thank you

Arun2536IT commented 1 year ago

Good afternoon, please help me to solve the issue.

time-format %T date-format %d/%b/%Y:%T %z log-format %h %^[%d:%t %^] "%m %U" %s %b "%R" "%u"

my log syntax :

Jul 26 13:13:49 3.109.145.218 157.35.22.23 10.0.1.39 - - [26/Jul/2023:13:13:49 +0530] "POST /ajaxPharmacy.php?sFlag=getClinicIdForDepartmentId HTTP/1.1" 200 4 "https://hims-dh.bhavyabiharhealth.in/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36"

the error i am receiving : root@ip-10-0-0-210:/var/log/dh# sudo goaccess /var/log/dh/dh_access_logs_combined.log -o dh_stats.html [PARSING /var/log/dh/dh_access_logs_combined.log] {0} @ {0/s} Cleaning up resources... ==787401== GoAccess - version 1.7.2 - Apr 1 2023 00:42:46 ==787401== Config file: /etc/goaccess/goaccess.conf ==787401== https://goaccess.io - hello@goaccess.io ==787401== Released under the MIT License. ==787401== ==787401== FILE: /var/log/dh/dh_access_logs_combined.log ==787401== Parsed 10 lines producing the following errors: ==787401== ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== Token 'Jul' doesn't match specifier '%h' ==787401== ==787401== Format Errors - Verify your log/date/time format

allinurl commented 1 year ago

@Arun2536IT 

goaccess access.log --log-format='%^:%^ %h %^[%x] "%r" %s %b "%R" "%u"' --datetime-format='%d/%b/%Y:%H:%M:%S %z'