eldy / AWStats

AWStats Log Analyzer project (official sources)
https://www.awstats.org
370 stars 120 forks source link

awstats 7.7 is slower than 7.5 #96

Open Fahad-Alsaidi opened 6 years ago

Fahad-Alsaidi commented 6 years ago

I upgrade from 7.5 to 7.7 and I notice that awstats takes longer time to finish process the log files. The number of visitors to my site almost the same as before. Is there any solution for that?

visualperception commented 6 years ago

how much longer and which country do you get most search engine visits from?

Fahad-Alsaidi commented 6 years ago

It takes more than 3 minutes and cpu goes to almost 100%. However, I delete *.tmp files in awstats data dir & it works fine.

visualperception commented 6 years ago

tmp files are exactly that. Awstats creates them during and then deletes them at end of run. If you have tmp files which aren't being deleted at end of run, then someting is wrong.

visualperception commented 6 years ago

Again, which google country do you get most visits from?

Fahad-Alsaidi commented 6 years ago

@visualperception here the most google country

visualperception commented 6 years ago

you could try moving the detection for saudi arabia and egypt to the the top of search engines file which may speed up things. The google.com is already towards the top and the google catchall should not be moved.

You need to move the entries in: SearchEnginesSearchIDOrder_list1 and SearchEnginesHashID

make sure they are in exact same position in both entries.

However, since google.com (catchall) is your biggest hit there is not much you can do about that unless you inspect your lour log file and find out what the google subdomain is that is going into catchall and add that google subdomain as an entry into your search engines file in all 4 lists and place it towards the top in the first two list as above.

my assumption is that entries towards the top of list are found more quickly than those further down the list.

The list was made much longer when I added all the google entries so there may have been a performance hit when they were added which was after version 7.5 I think.

Fahad-Alsaidi commented 6 years ago

@visualperception I did what you suggest but no improvement. Awstats takes long time to process this step:

Build lasthosts page

visualperception commented 6 years ago

Sorry but thats all I can suggest.

brentil commented 6 years ago

You might want to use a merge/compare tool to check your conf file against the latest awstats.model.conf from 7.7 to make sure you don't have any odd settings or missing configuration if it's been a while since you've built those original conf files.

How much longer are your scans actually taking from 7.5 to 7.7? You can expect there to be some increase in time since the databases of bots/OS/browser/etc have increased so parsing them might take some extra time.

You may also want to check how old your Perl is too.

visualperception commented 6 years ago

just out of curiosity, is it taking longer when you are running it in debug mode or when you have debug switched off?