matomo-org / matomo-log-analytics

Import any kind of server logs in Matomo for powerful log analytics. Universal log file parsing and reporting.
https://matomo.org/log-analytics/
GNU General Public License v3.0
225 stars 118 forks source link

IndexError: list index out of range #158

Open mcnesium opened 7 years ago

mcnesium commented 7 years ago

Sometimes I get this error messages while parsing the log file:

Traceback (most recent call last):
 File "/home/me/www/analytics.mcnesium.com/misc/log-analytics/import_logs.py", line 2371, in <module>
   config = Configuration()
 File "/home/me/www/analytics.mcnesium.com/misc/log-analytics/import_logs.py", line 917, in __init__
   self._parse_args(self._create_parser())
 File "/home/me/www/analytics.mcnesium.com/misc/log-analytics/import_logs.py", line 897, in _parse_args
   self.options.piwik_token_auth = self._get_token_auth()
 File "/home/me/www/analytics.mcnesium.com/misc/log-analytics/import_logs.py", line 999, in _get_token_auth
   return credentials[1]
IndexError: list index out of range

The following script gets log files via ssh from other servers and passes them on to the importer:

PIWIKID="$1";
USERHOST="$2";
LOGFILE="$HOME/tmp/$USERHOST.log";
PIWIKURL="https://analytics.mcnesium.com";
IMPORTSCRIPT="$HOME/www/analytics.mcnesium.com/misc/log-analytics/import_logs.py";

if scp -i "$HOME/.ssh/rsync_rsa" "$USERHOST:www/logs/access_log" "$LOGFILE" > /dev/null ; then
        if [ -s "$LOGFILE" ]; then      # if it's a non-empty file
                PARAMS[0]="--url=$PIWIKURL";
                PARAMS[1]="--enable-reverse-dns";
#               PARAMS[2]="--enable-bots";
#               PARAMS[3]="--enable-http-errors";
                PARAMS[4]="--enable-http-redirects";
                PARAMS[5]="--enable-static";
                if /usr/bin/python "$IMPORTSCRIPT" $(printf '%s ' "${PARAMS[@]}") "--idsite=$PIWIKID" "$LOGFILE" > /dev/null ; then
                        # success
                        rm $LOGFILE;
                else
                        # fail
                        KEEPLOGFILE="$(dirname $LOGFILE)/$(date +'%Y%m%d%H%M%S').log";
                        mv $LOGFILE $KEEPLOGFILE;
                fi
        else
                echo "$LOGFILE doesn't exist or has size zero.";
        fi
fi

The above error message then comes via mail sent by the cron job. The logfile created in this session can be downloaded here. Why is it doing this sometimes? The cron job runs hourly for multiple log files originating on different servers, pulling all of them into this piwik instance. Mails with those error messages appear randomly every other day, no pattern observable. Any ideas?

mattab commented 7 years ago

Sorry no idea. Does anyone else experience this issue?

Findus23 commented 5 years ago

The error is because misc/cron/updatetoken.php doesn't return the proper response. To circumvent this, I'd recommend to either use --login="admin" --password="password" or --token-auth directly. This way the script doesn't depend on PHP.