Feature request: Speed up import_password

[GoogleCodeExporter]

I read around that pyrit provide to garantee that all password are unique
and respect standard size: this feature slow down the import_password task
(each password must be compared with existing password already present in 
in database).
Because I am *sure* about the fact that passwords of my list are unique and
respect the size, can you add a flag that allow me to skip this control? It
will be speed up - I suppose - tremendously the import process of long list
of passwords.
Because the code already exist, it will be - I think - easy and fast to add
a flag to exclude that portion of code. I will strongly appreciate, because
it will save me about 10% off all computation time. ( I cannot use -i and
-o options present in 0.2.5 svn because in case of crash I will loose all
the job done as reported into issue #76 comments #12 and #14.
I tried to read and manipulate storage.py by myself, but I am not a python
coder, so I am quite sute i will have success only to make disasters into
code: in am sure only to eliminate size test (>8 or <63) but I duno where
to work to eliminate unique test.

Original issue reported on code.google.com by pyrit.lo...@gmail.com on 19 Dec 2009 at 1:38

[GoogleCodeExporter]

storage.py:

passwd = passwd.strip() ###define the passwd
if len(passwd) < 8 or len(passwd) > 63: ###here check the lenght, if not valid
 return #quit the function, so
pw_h1 = PasswordStore.h1_list[hash(passwd) & 0xFF] ###when valid, define pw_h1 
and
start another function
pw_bucket = self.pwbuffer.setdefault(pw_h1, set()) ###here another check and 
update
of passwords set
pw_bucket.add(passwd) ###here adds the passwd

try to follow what does .setdefault(,) and the entire class 
PasswordStore(object), but
"""EssidStore and PasswordStore are the primary storage classes. Details of
   their implementation are reasonably well hidden behind the concept of
   key:value interaction.
"""like lukas wrote at the beginning
It seems the unique test is done by python builtin functions, that should be
reasonable fast, however

Original comment by masterzorag on 6 Jan 2010 at 4:00

[GoogleCodeExporter]

hi materzorag.
even if "unique test is done by python builtin functions, that should be
reasonable fast" it will be always to much slow than not do the test at all.
Consider that import 240-250 Million of password (I use pyrit 0.2.4) take 50 
minutes.
Export in cowpatty format it needs only 12 minutes (and export does a bigger 
I/O than
import password). It will be great to reduce that 50 minutes to 5-10 minutes, 
but i
duno how to eliminate that unique test. I can only operate on exaustive way: 
compare
an imported number of passwords before and after I eliminate a function... at 
the
end, I should find the right function to eliminate.

Original comment by pyrit.lo...@gmail.com on 6 Jan 2010 at 10:18

[GoogleCodeExporter]

I'll implement this later on. Please don't guess.

Original comment by lukas.l...@gmail.com on 7 Jan 2010 at 7:22

[GoogleCodeExporter]

fixed in r202

Original comment by lukas.l...@gmail.com on 10 Jan 2010 at 8:07

• Changed state: Fixed