search

darold / squidanalyzer

Squid Analyzer parses Squid proxy access log and reports general statistics about hits, bytes, users, networks, top URLs, and top second level domains. Statistic reports are oriented toward user and bandwidth control.
http://squidanalyzer.darold.net/
126 stars 36 forks source link

Cannot parse data (index.html empty) #114

Closed Akaban closed 8 years ago

Akaban commented 8 years ago

Hello Darold,

First of all I would like to thank you for this tool of yours that will be very useful to me, I installed squidanalyzer and read carefuly the man but there's no way I can make squidanalyzer to parse my logs.

These are like 300M and squid-analyzer finish in few seconds, producing nothing as the result.

I also tried with a small piece of log which is

Dec 4 03:31:46 pharaon (squid-1): 1449196306.934 45 172.17.20.24 TCP_MISS/302 2253 GET http://t4.liverail.com/? - HIER_DIRECT/179.60.192.12 text/html Dec 4 03:31:46 pharaon (squid-1): 1449196306.943 66096 172.17.6.82 TCP_MISS/200 6016 CONNECT v2.auc.avira.com:443 - HIER_DIRECT/52.19.80.30 - Dec 4 03:31:46 pharaon (squid-1): 1449196306.944 45 172.17.20.24 TCP_MISS/302 2240 GET http://t4.liverail.com/? - HIER_DIRECT/179.60.192.12 text/html Dec 4 03:31:46 pharaon (squid-1): 1449196306.954 46 172.17.20.24 TCP_MISS/302 2187 GET http://t4.liverail.com/? - HIER_DIRECT/179.60.192.12 text/html Dec 4 03:31:46 pharaon (squid-1): 1449196306.957 46 172.17.20.24 TCP_MISS/302 2231 GET http://t4.liverail.com/? - HIER_DIRECT/179.60.192.12 text/html

I thought that the issue was the date at the beginning (which has been added by syslog I think) but even with these removed it's still not working.

Here's what /usr/bin/squid-analyzer -d -r test1.log says:

SquidAnalyzer version 6.3 No new log registered... DEBUG: the log statistics gathering took: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU) Building HTML output into /tmp/test DEBUG: generating HTML output took: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU) DEBUG: total execution time: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)

Do you have any clues about this particular issue?

Thanks by advance.

darold commented 8 years ago

Hi,

Thanks for your interest in this project. Unfortunately SquidAnalyzer can only parse native, common or combined log format.

Best regards

Akaban commented 8 years ago

Hey,

Thanks for your answer, but my logs aren't the squid native format? I made no modification about the log format so it should be the default therefore the native format. I forgot to say I was using Squid3

tierpod commented 8 years ago

@Akaban

You have two date fields. Try remove 'Dec 4 03:31:46 pharaon (squid-1): ' from log:

1449196306.934 45 172.17.20.24 TCP_MISS/302 2253 GET http://t4.liverail.com/? - HIER_DIRECT/179.60.192.12 text/html
darold commented 8 years ago

I think you are logging squid through syslog this is why all lines start with "Dec 4 03:31:46 pharaon (squid-1): ". Native squid log format starts after that. You must replace syslog by daemon in your squid3.conf file. For example:

access_log daemon:/var/log/squid3/access.log squid

should fix your issue.