OverflowError

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Import a 30 Gigabyte password file
2.
3.

What is the expected output? What do you see instead?
Error as follow:

localoser@bt:~$ sh input_many_files_to_pyrit.sh
Pyrit 0.3.1-dev (svn r280) (C) 2008-2010 Lukas Lueg http://pyrit.googlecode.com
This code is distributed under the GNU General Public License v3+

Connecting to storage at 'file://'...  connected.
2147400000 lines read (427354.0 lines/s)... Traceback (most recent call last):
  File "/usr/bin/pyrit", line 6, in <module>
    pyrit_cli.Pyrit_CLI().initFromArgv()
  File "/usr/lib/python2.5/site-packages/pyrit_cli.py", line 116, in initFromArgv
    func(self, **options)
  File "/usr/lib/python2.5/site-packages/pyrit_cli.py", line 355, in import_passwords
    for i, line in enumerate(reader):
OverflowError: enumerate() is limited to LONG_MAX items
localoser@bt:~$

What version of the product are you using? On what operating system?
BackTrack4 (Ubuntu based)

Please provide any additional information below.

Original issue reported on code.google.com by henrique...@gmail.com on 12 Mar 2011 at 11:47

[GoogleCodeExporter]

Maybe I am stupid, but not problem backtrack4 r2 support only 32bit os?

Original comment by andor...@gmail.com on 13 Mar 2011 at 8:35

[GoogleCodeExporter]

Maybe, am using 32bits cause only have 2.5 GB of RAM. I have successfully 
imported an 11 GB text file but not this one:

-rwxrwxrwx 1 root root 38793140809 2009-04-08 06:14 30GB.txt

I'm currently splitting it and importing it, will report later.

It seems that pyrit (like every software tbh) doesn't scale well with huge text 
files because it's getting slower and slower when merging other files into the 
database that was fed with several dozens of gigabytes from other password 
files. It's not a pyrit issue but rather an architectural one, the hardware I'm 
using it's also pretty outdated from today's standards, who the bleep still 
uses a Pentium 4 ?! :)

Original comment by henrique...@gmail.com on 13 Mar 2011 at 8:52

[GoogleCodeExporter]

You need to split such large files on a 32bit system. The error comes directly 
from the Python-interpreter.

Original comment by lukas.l...@gmail.com on 13 Mar 2011 at 5:58

• Changed state: WontFix