AaronVanGeffen / AwstatsParser

PHP script for parsing and merging awstats data files
BSD 2-Clause "Simplified" License
11 stars 6 forks source link

A non-numeric value encountered warnings in output file #5

Open foovax opened 6 years ago

foovax commented 6 years ago

The tool seemed to get the job done and I appreciate the time and effort put into this. I'm not sure if there is an "official" way of merging files currently but I noticed that the output from the script was old:

AWSTATS DATA FILE 6.9 (build 1.925)

The two files being merged are: AWSTATS DATA FILE 7.4 (build 20140726) AWSTATS DATA FILE 7.6 (build 20161204)

At the top of the output there were two Warnings:

Warning: A non-numeric value encountered in /home/private/AwstatsParser-master/AwstatsParser/AwstatsMerger.php on line 272 Warning: A non-numeric value encountered in /home/private/AwstatsParser-master/AwstatsParser/AwstatsMerger.php on line 272

With the warnings the file could not be parsed by awstats giving an error:

History file '/tmp/awstats122017.txt' is to old (version '0'). This version of AWStats is not compatible with very old history files. Remove this history file or use first a previous AWStats version to migrate it from command line with command: awstats.pl -migrate="/tmp/awstats122017.txt"

After removing the warnings from each file I performed the migrate as instructed: # awstats.pl -migrate="/tmp/awstats122017.txt"

Which then output: AWSTATS DATA FILE 7.7 (build 20180105)

The output was able to parse as expected and all the data totals seemed accurate after parsing in awstats.cgi.

There was no additional analysis performed to determine what rows precisely are causing this non-numeric values but I suspect due to data file changes between the older and newer versions.

Line 272 requires checking of $row prior to the for loop to prevent empty $row from reaching foreach which is throwing the warning:

foreach ($row as $num => $stats) $this->data[$section_name][$key][$num] += $stats; along with suppressing script output directly into data files and routing that to STDERR.

I assume this to be a non trivial modification so I write this in the event someone else comes along experiencing a similar issue.

AaronVanGeffen commented 6 years ago

Thank you for your report. As you've surmised, the issue with Awstats is PHP error messages ending up in the resulting file — unfortunately, they are printed to stdout instead of stderr. We can work around this by having the script output directly to an output file, rather than relying on output redirection, but that doesn't solve the underlying issue.

The issue you're experiencing sounds similar to that described in #3, which I wasn't able to reproduce in the end. I'd be very interested to learn the cause, however.

Could you replace the following lines in AwstatsMerge.php (271-273):

            foreach ($row as $num => $stats)
                $this->data[$section_name][$key][$num] += $stats;
        }

with these:

            foreach ($row as $num => $stats)
            {
                if (is_numeric($stats))
                    $this->data[$section_name][$key][$num] += $stats;
                else
                {
                    echo 'Section name: ', $section_name, "\n",
                        'Key: ', $key, "\n",
                        'Num: ', $num, "\n",
                        'Stats: ', $stats, "\n";
                }
            }
        }
        exit;    

This change should make the script terminate after outputting details on the offending lines. (Note: the exit statement should be the last statement in the function.)

Please find the offending lines in the files you're trying to merge, and report them here.

foovax commented 6 years ago

Ha! indeed it's a duplicate of #3 and I apologize.

I had to modify the provided block of code but was able to get output:

Section name: SIDER_404 Key: /robots.txt Num: 1 Stats: - AWSTATS DATA FILE 6.9 (build 1.925)

foovax commented 6 years ago

And from another file similar output:

Section name: SIDER_404 Key: /wp-content/plugins/font-uploader/font-upload.php Num: 1 Stats: - AWSTATS DATA FILE 6.9 (build 1.925)

AaronVanGeffen commented 6 years ago

Thanks, that's a big help! Odd that it would have a hyphen in that position. Something to account for in the script, for sure.