Issue with Static Requests

[oddjobz]

Hi, I 'm using the real-time html option with websockets, generally works great. However the static request panel remains empty with 20,000 entries + in the system. I've tried adding and tweaking variations of "static-file" to .goaccessrc and the command line, but nothing seems to have any effect. (complete mix of content from html with no ext, css, js, png, jpeg etc ..)

I'm was expecting it to either be populated from a default extension list or from a list of extensions I supply, do I misunderstand what it's supposed to do?

If so, is there some special requirement needed to populate it? (literally everything else seems fine ...)

[allinurl]

That's a great question! There’s a standard list of static files, but if you provide a list in the config file, it will override the defaults and use the specified files instead. Please ensure they are formatted as follows:

static-file .css
static-file .js
static-file .jpg
static-file .png
static-file .gif
static-file .ico
static-file .jpeg
static-file .pdf
static-file .csv
static-file .mpeg
static-file .mpg
static-file .swf
static-file .woff
static-file .woff2
static-file .xls
static-file .xlsx
static-file .doc
static-file .docx
static-file .ppt
static-file .pptx
static-file .txt
static-file .zip
static-file .ogg
static-file .mp3
static-file .mp4
static-file .exe
static-file .iso
static-file .gz
static-file .rar
static-file .svg
static-file .bmp
static-file .tar
static-file .tgz
static-file .tiff
static-file .tif
static-file .ttf
static-file .flv
static-file .dmg
static-file .xz
static-file .zst

[oddjobz]

Cool, so my problem is that without tampering, nothing is appearing in my static requests html panel. If I run from the command line, I can immediately see .js files in the "requested files" table, but the "static files" counter at the top reads zero. I've tried adding "static-file .js" to .goaccessrc, no change .. any ideas?

[allinurl]

Are you using the latest version? What does the output of goaccess --dcf say? You might also want to try using the --no-global-config option.

[oddjobz]

Ok, so;

# goaccess --dcf /var/log/nginx/vhosts_access.log
/root/.goaccessrc

Adding --no-global-config, no discernible difference. I have it set up to produce a separate html output for each vhost, which runs under a user account, then I have a "global" option that consumes all logging information to provide an overview of all sites. (I'm testing with the latter, but all suffer the same issue)

The global uses root's .goaccess, the per vhosts uses the .goaccess from the user account. Both are the same;

# cat .goaccessrc 
log-format %^ %^ %v %h - - [%d:%t %^] "%r" %s %b "%R" "%u"
date-format %d/%b/%Y
time-format %H:%M:%S

(this is parsing input from syslog based on data received from nginx via it's syslog logging module) I have 11 instances of goaccess running in --daemonize mode, as I say all look great, except ...

[allinurl]

Just to clarify, root's .goaccessrc contains only:

log-format %^ %^ %v %h - - [%d:%t %^] "%r" %s %b "%R" "%u"
date-format %d/%b/%Y
time-format %H:%M:%S

In that case, you'll need to include the extensions I mentioned earlier.

[oddjobz]

My launch string looks like this (sensitive information redacted with sid/user/pass);

goaccess /var/log/nginx/input.log \
        -o /home/mms_weblogs/.local/mms_weblogs/vhosts/sid/index.html \
        --db-path /home/mms_weblogs/.local/mms_weblogs/cache/sid \
        --ssl-cert /home/mms_weblogs/.local/mms_weblogs/ssl.cert 
        --ssl-key /home/mms_weblogs/.local/mms_weblogs/ssl.key 
        --ws-url wss://user:pass@weblogs.makemestatic.com:443/ws \
        --port 9999 \
        --pid-file /tmp/sid.pid \
        --html-prefs {"theme":"bright", "perPage": 5, "layout": "horizontal"} \
        --ignore-panel REQUESTS_STATIC \
        --ignore-panel KEYPHRASES \
        --ignore-panel ASN \
        --real-time-html -qa \
        --persist --restore \
        --anonymize-ip --keep-last 30 --daemonize

But again, the CLI seems to see the same issue.

$ goaccess --version
GoAccess - 1.9.3.
For more details visit: https://goaccess.io/
Copyright (C) 2009-2024 by Gerardo Orellana

Build configure arguments:
  --enable-utf8
  --enable-geoip=mmdb
  --with-getline
  --with-openssl

[oddjobz]

Ok, so I added your complete list of extensions, I now see a non-zero in static files on the CLI, so that looks like it did something .. previously I only added .js and .css which didn't seem to work, let me check what it's done ...

[oddjobz]

Ok, so this is just loading images ... no CSS or JS, and I can see it is using CSS and JS .. however typically these are versioned with ?ver= , is this what's stopping them from being listed?

[oddjobz]

Interesting ... despite the TX amount header in static files showing transfer, each of the files listed in the tables (on all pages of the table) is showing 0 byte .. is there any way this sounds right? Notably, clicking on the TX amount header doesn't change the sort order, which also seems to confirm all entries are zero?

[oddjobz]

Ok, not sure what's going on but it's started to register non-zero sizes, so whatever it was I seem to have fixed .. tempted to think it was registering cached hits (as zero) but I specifically cleared all the cache entries before testing it .. still, looks good. (many thanks!)

[allinurl]

Glad it's working for you. Feel free to reopen it if needed.