I don't understand about nx_util.py-3.0

[GoogleCodeExporter]

i  builded a report like this:
=========================================
$ nx_util.py -l /www/logs/nginx_error.log  -o -H log1.html
Deleting old database :/tmp/nx_utilnaxsi_sig
List of imported files :['/www/logs/nginx_error.log']
Importing file /www/logs/nginx_error.log
        Successful events :1544
        Filtered out events :0
        Non-naxsi lines :0
        Malformed/incomplete lines 51526
End of db commit...
Count (lines) success:1544
Traceback (most recent call last):
  File "/usr/local/bin/nx_util.py", line 105, in <module>
    base_rules, opti_rules = wl.opti_rules_back()
  File "/usr/local/lib/python2.6/site-packages/nx_lib/nx_whitelists.py", line 86, in opti_rules_back
    if len(r['var_name']) > 0:
IndexError: No item with that key
=========================================
and then i get the error message .
there is  something ?

Original issue reported on code.google.com by Chas...@gmail.com on 26 Apr 2013 at 3:39

[GoogleCodeExporter]

Hello,

This is a rather strange issue, and might be caused by incorrect/incomplete 
naxsi lines (even nx_util tries to handle those).
In order to understand better what's going on, could you please add the 
following line in nx_lib/nx_whitelists.py at line 86 (just before the "if 
len(r['var_name']) > 0:) :
print(r)

This will print the exceptions that are extracted from database, and you should 
look at the last print before the exception.
The output should allow you to find back the faulty line as well (you will have 
ID, url etc.). Once you've spotted it, please send it to us, so we can fix this 
exception.

thanks,

Original comment by ori...@gmail.com on 26 Apr 2013 at 7:46

[GoogleCodeExporter]

Original comment by ori...@gmail.com on 13 May 2013 at 9:36

• Changed state: Invalid

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Hi,

I got this error too. And when I add print(r), I got:
(100, 1001, 'ARGS', 'vulnarg', '//foobar', 1, 1, 100)
Traceback (most recent call last):
  File "/usr/local/python-2.7.5/bin/nx_util.py", line 117, in <module>
    base_rules, opti_rules = wl.opti_rules_back()
  File "/usr/local/python-2.7.5/lib/python2.7/site-packages/nx_lib/nx_whitelists.py", line 94, in opti_rules_back
    if len(r['var_name']) > 0:
IndexError: No item with that key

It looks like the results from sqlite missed something? It ought to be a dict, 
but It just a array?

Original comment by bandnew...@gmail.com on 13 Jun 2013 at 8:37

[GoogleCodeExporter]

Hi guy,

I guess this is which backend is used.
I have the same error with
- nx_util 1.0 or 0.3
- on centos5
- with python2.6

I guess the problem is due to sqlite

When nx_util fetch data from the database:
for req in opti_select_DESC:
            res = self.wrapper.execute(req)
            #res = self.wrapper.getResults()
            for r in res:

I can get the first field of a line:
print r[0] 
but I cannot do it via column name:
print r['exception_id']

I don't have tested yet with a mysql backend.

Original comment by nicolas....@gmail.com on 20 Jun 2013 at 9:22

[GoogleCodeExporter]

I answer to myself.

the request is 
select  count(*) as ct, e.rule_id, e.zone, e.var_name, u.url, count(distinct 
c.peer_ip) as peer_count, (select count(distinct peer_ip) from connections) as 
ptot, (select count(*) from connections) as tot from exceptions as e, urls as 
u, connections as c where c.url_id = u.url_id and c.id_exception = 
e.exception_id GROUP BY u.url, e.var_name,e.zone, e.rule_id HAVING (ct) > 
((select count(*) from connections)/1000)

so in fact, the field should not be r['var_name'] but r['e.var_name']

HTH

Original comment by nicolas....@gmail.com on 20 Jun 2013 at 9:35

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

If people are interested, here is my "patch" for 
nx_util/nx_lib/nx_whitelists.py.
It begins at line 91

        for req in opti_select_DESC:
            res = self.wrapper.execute(req)
            #res = self.wrapper.getResults()
            for r in res:
              try:
                if len(r['e.var_name']) > 0:
                    self.try_append({'url': r['u.url'], 'rule_id': r['e.rule_id'], 'zone': r['e.zone'],  'var_name': r['e.var_name'],
                                     'hcount':  r['ct'], 'htotal': r['tot'], 'pcount':r['peer_count'], 'ptotal':r['ptot'],
                                     'pratio': round((r['peer_count'] / float(r['ptot'])) * 100,2),
                                     'hratio': round((r['ct'] / float(r['tot'])) * 100,2)
                                     })
                else:
                    self.try_append({'url': r['u.url'], 'rule_id': r['e.rule_id'], 'zone': r['e.zone'], 'var_name': '',
                                     'hcount': r['ct'],  'htotal': r['tot'], 'ptotal':r['ptot'],
                                     'pratio': round((r['peer_count'] / float(r['ptot'])) * 100,2),
                                     'hratio': round((r['ct'] / float(r['tot'])) * 100,2),
                                     'pcount':r['peer_count']})
              except IndexError:
                pass
        return self.base_rules, self.final_rules

Original comment by nicolas....@gmail.com on 20 Jun 2013 at 9:59

[GoogleCodeExporter]

^Thanks for the patch man, I wonder why naxsi dev still not fixing this error.

Original comment by o...@sinonet.ph on 5 Sep 2013 at 4:06