allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
17.78k stars 1.09k forks source link

Counters overlaps in parsing progress output #2637

Closed rc5hack closed 3 months ago

rc5hack commented 4 months ago

While parsing big (5kk lines, 1Gb+) logfile, counters in progress output overlaps each other:

 [PARSING /path/to/file.log] {413696} @ {17237/s}d] {0} @ {0/s}
 [PARSING /path/to/file.log] {836023} @ {17787/s}
 [PARSING /path/to/file.log] {1234948} @ {17393/s}
 [PARSING /path/to/file.log] {1623040} @ {17266/s}
 [PARSING /path/to/file.log] {1987584} @ {16843/s}
 [PARSING /path/to/file.log] {2343602} @ {16621/s}
 [PARSING /path/to/file.log] {2703367} @ {16377/s}
 [PARSING /path/to/file.log] {3066806} @ {16302/s}
 [PARSING /path/to/file.log] {3433472} @ {16188/s}
 [PARSING /path/to/file.log] {3821895} @ {16263/s}
 [PARSING /path/to/file.log] {4206259} @ {16240/s}
 [PARSING /path/to/file.log] {4574208} @ {16220/s}@ {16245/s}
 [PARSING /path/to/file.log] {4954409} @ {16243/s}6225/s}
 [PARSING /path/to/file.log] {5348178} @ {16255/s}/s}
 [PARSING /path/to/file.log] {5660451} @ {16265/s}
WebSocket server ready to accept new client connections

Looks like there are more lines that shown, and they are competing some way 🤷🏻‍♂️

I can reproduce this problem in versions 1.8.1 and 1.9.1.

Running goaccess with docker compose, with the following config:

version: '2.4'
services:
   goaccess:
    container_name: goaccess
    image: allinurl/goaccess:1.9.1
    volumes:
      - ./geodb:/opt/geodb/:ro
      - ./www/:/opt/www/:rw
      - /path/to/logs/:/opt/logs/:ro
    command:
      - /opt/logs/file.log
      - --geoip-database=/opt/geodb/GeoLite2-City.mmdb
      - --agent-list
      - --with-output-resolver
      - --real-time-html
      - --html-refresh=1
      - --log-format=VCOMBINED
      - --output=/opt/www/index.html
      - --html-report-title=logs analysis
      - --ws-url=wss://${DOMAIN_NAME}:443/ws
      - --origin=https://${DOMAIN_NAME}
      - --enable-panel=VISITORS
      - --enable-panel=REQUESTS
      - --enable-panel=REQUESTS_STATIC
      - --enable-panel=NOT_FOUND
      - --enable-panel=HOSTS
      - --enable-panel=OS
      - --enable-panel=BROWSERS
      - --enable-panel=VISIT_TIMES
      - --enable-panel=VIRTUAL_HOSTS
      - --enable-panel=REFERRERS
      - --enable-panel=REFERRING_SITES
      - --enable-panel=KEYPHRASES
      - --enable-panel=STATUS_CODES
      - --enable-panel=REMOTE_USER
      - --enable-panel=GEO_LOCATION
      - --html-prefs={"theme":"darkPurple","perPage":10,"layout":"vertical"}
    restart: unless-stopped
    cpus: 1.00
    mem_limit: 256m
allinurl commented 3 months ago

Just so I understand the issue here, are you referring to the {16220/s}@ {16245/s} {16243/s}6225/s}, etc output?

allinurl commented 3 months ago

Assuming that was what you were referring to, I went ahead and pushed a possible fix to it. Please build from development and let me know if that solves the issue for you.

rc5hack commented 3 months ago

I wasn't able to build from development (#2657), but if the newer docker image in dockerhub (allinurl/goaccess:latest - pushed 28.03.2024) is also built from development, it doesnt'f fix problem.

How it looks with allinurl/goaccess:1.9.1: console

How it looks with allinurl/goaccess:latest: console

allinurl commented 2 months ago

Thanks for sharing that. What shell and terminal are you using? Also, do you hit Enter, or does it just show a new line every few seconds?

rc5hack commented 2 months ago

I don't hitting Enter, all lines on screenshots appears on their own.

Problem reproduces with zsh 5.8 (5.8-3ubuntu1.1), bash 5.1 (5.1-2+deb11u1), dash 0.5.11 (Debian 11).

Problem reproduces when ssh-ing to server via Windows Terminal 1.18.10301.0, via PuTTY 0.76.

docker compose version is 2.20.0.