search

allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.
https://goaccess.io
MIT License
18.6k stars 1.11k forks source link

goaccess built with LTO is super slow #1876

Open Titaniumtown opened 4 years ago

Titaniumtown commented 4 years ago

So it takes my server 2 minutes to run goaccess. I check to see if it's bottle necked, it's not. CPU is like at 1% percent, disk IO is like nothing, but it's still super slow. Any thoughts?

Edit: The issue is related to compiling goaccess with LTO (link time optimizations).

LucaNerlich commented 4 years ago

how large are your access_log files?

Titaniumtown commented 4 years ago

a little over 1mb

LucaNerlich commented 4 years ago

ok, that is tiny. Have you tried to copy the log file to another machine and run goaccess again? I assume it is some kind of infrastrucutre issue you are facing.

Titaniumtown commented 4 years ago

Takes 2.8 sec on my main pc

Titaniumtown commented 4 years ago

what I run: goaccess /mnt/tb_hdd/logs/access_log_apache.txt --log-format=COMBINED --num-test=0 -o /mnt/hdd/http_share/log.html -a -e 107.145.175.222 --real-os

Titaniumtown commented 4 years ago

It's on a raspberry pi 4 overclocked to 2ghz, if that matters. But it shouldn't be bottlenecked.

Titaniumtown commented 4 years ago

Is there some way for me to see what the slowdown is?

LucaNerlich commented 4 years ago

you could clone the repo and try to debug it

Titaniumtown commented 4 years ago

wait... it seems if i use the official release, it finishes in 5 seconds. but if i use the binary created when i compile it from source. it takes like 1 min

Titaniumtown commented 4 years ago

that's strange....

Titaniumtown commented 4 years ago

I'm going to try something....

Titaniumtown commented 4 years ago

Fixed it. I compiled goaccess with LTO before. Now that i disabled it, it works. weird

Titaniumtown commented 4 years ago

@allinurl Do you have any idea why goaccess would be slower when built with LTO?

Titaniumtown commented 4 years ago

Is this something common with compiling things with LTO?

allinurl commented 4 years ago

To be honest, I don't have a concrete answer, like @LucaNerlich said, you may need to debug to find out where exactly is the bottleneck. I'm guessing somewhere it's over optimizing and performing additional cycles, maybe branch instruction throughput...

Can you post how you are building goaccess?

Titaniumtown commented 4 years ago

I just use the aur package goaccess-git, then i put some flags in makepkg.conf