allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
18.53k stars 1.11k forks source link

Version 1.6.3 introduces "Token for specifier is NULL" error #2717

Open Proper-Job opened 2 months ago

Proper-Job commented 2 months ago

When I use v1.6.2 this config is able to parse these logs:

Config

# ~/.goaccessrc
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %D %h %^ %^ [%d:%t %^] "%r" %s %b "%R" "%u" "~h" "%^"

Logs

336 10.5.176.14 - - [16/Sep/2024:00:01:01 +0200] "GET /?site=pim HTTP/1.1" 200 8803 "-" "curl/8.0.1" "10.1.232.31" "example.com"
284 10.5.176.14 - - [16/Sep/2024:00:01:01 +0200] "GET /?site=pim HTTP/1.1" 200 8803 "-" "curl/8.0.1" "10.1.224.217" "example.com"

Starting on version 1.6.3 (all the way up to 1.9.3) parsing fails with the same config on the same logs with this error message:

goaccess access_log -o report.html

[SETTING UP STORAGE access_log] {0} @ {0/s}
Cleaning up resources...
==90724== GoAccess - version 1.6.3 - Sep 18 2024 12:02:28
==90724== Config file: /Users/me/.goaccessrc
==90724== https://goaccess.io - <hello@goaccess.io>
==90724== Released under the MIT License.
==90724==
==90724== FILE: access_log
==90724== Parsed 2 lines producing the following errors:
==90724==
==90724== Token for '%h' specifier is NULL.
==90724== Token for '%h' specifier is NULL.
==90724==
==90724== Format Errors - Verify your log/date/time format

Build config

# MacOS X 14.6.1
./configure --enable-utf8 --enable-geoip=mmdb --with-openssl --prefix=$HOME/bin/goaccess

Your build configuration:

  Prefix         : /Users/me/bin/goaccess
  Package        : goaccess
  Version        : 1.6.3
  Compiler flags :  -pthread
  Linker flags   : -lncurses -lmaxminddb -lcrypto -lssl -lpthread -lintl -lintl
  UTF-8 support  : yes
  Dynamic buffer : no
  Geolocation    : GeoIP2
  Storage method : In-Memory with On-Disk Persistent Storage
  TLS/SSL        : yes
  Bugs           : hello@goaccess.io
allinurl commented 2 months ago

There have been some updates to the XFF fields and their parsing. You can find more details in the release notes here: https://goaccess.io/release-notes#release-1.6.3. Be sure to check the man page for examples. Depending how your IPs are delimited, you could use for instance,

goaccess access.log --log-format='%D %^ %^ %^ [%d:%t %^] "%r" %s %b "%R" "%u" "~h{,}" "%v"' --date-format=%d/%b/%Y --time-format=%T