crimson-unicorn / parsers

GNU General Public License v2.0
20 stars 18 forks source link

Exception AttributeError: "'list' object has no attribute 'clear'" in 'rocksdb._rocksdb.DB.__dealloc__' #1

Open annygakh opened 5 years ago

annygakh commented 5 years ago

Hi! I have encountered the following error when trying to run provparser on a clean installation of Ubuntu 18.04 (run in a virtual machine).

[WARNING] Use '-t fivedirections' only for ta1-fivedirections-e3-official data. Exit now otherwise...
[+] initiating parser...
[i] multiprocessing support is on for processing but not scanning
[+] processing regular JSON files in directory data/...
[i] multiprocesses processing regular JSON files
[+] setting up database python1.log.db in current directory...
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/anny/rocksdb/build/parsers/cdm/ProvParser/provparser/provparser", line 96, in process
    with open(os.path.join(args.input, fn), 'r') as fileobj:
IOError: [Errno 13] Permission denied: 'data/python1.log'
AttributeError: 'list' object has no attribute 'clear'
Exception AttributeError: "'list' object has no attribute 'clear'" in 'rocksdb._rocksdb.DB.__dealloc__' ignored
[+] node parsing is done
[+] parsing files again to output final results.
[i] multiprocessing support is on...
[+] processing regular JSON files in directory data/...
[i] opening output file camflow-out.txt for writing...
Traceback (most recent call last):
  File "/home/anny/rocksdb/build/parsers/cdm/ProvParser/testenv/bin/provparser", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/home/anny/rocksdb/build/parsers/cdm/ProvParser/provparser/provparser", line 371, in <module>
    fileobj = open(os.path.join(args.input, sortedfilenames[0]), 'r')
IOError: [Errno 13] Permission denied: 'data/python1.log'
AttributeError: 'list' object has no attribute 'clear'
Exception AttributeError: "'list' object has no attribute 'clear'" in 'rocksdb._rocksdb.DB.__dealloc__' ignored

Do you happen to have any suggestions on how to fix this, please?

It seems like somewhere in rocksdb they use a python3 function which is not compatible with the python2. In another VM I have also tried to make python3 the default version, and installed everything, but when I would try to run provparser I would get an error along the lines of prepare module is not found (I don't have the exact error text but I can redo the installation and paste the error code here)

HermitSun commented 3 years ago

I switched to python 3.5 and pip 18.1, and it can work. But there are so many errors like this:

[error]  skipping this problematic line:264302841624046474      10453172175784923055    16880477119410153797:10290171809009782831:790722838836215523:1521784789000907344