Speed improvements with -s|--skip

[Sc00bz]

Benchmarks with "time ./pp64.bin -l 10 -s **\ < rockyou-top50k.txt"

New skip 100,000,000: 0.095s Current skip 100,000,000: 2.535s

New skip 1,000,000,000: 0.115s Current skip 1,000,000,000: 24.583s

New skip 100,000,000,000: 10.520s

I'm still a little disappointed in the new version's speed. There is one more thing that can be done. Initializing dbentries[]->elems_pos and dbentries[]->elems_buf[db_entries[*]->elems_pos]->ks_pos before the main loop. I have some ideas as to doing this semi-efficiently. I'll probably do that tonight.

[jsteube]

Btw, the output performance is increased by that patch, too

old:

$ time ./pp64.bin -l 100000000 -o /dev/null < rockyou.txt 

real    0m13.033s
user    0m12.884s
sys     0m0.148s

new:

$ time ./pp64.bin -l 100000000 -o /dev/null < rockyou.txt 

real    0m12.109s
user    0m11.924s
sys     0m0.184s

Great work!

[magnumripper]

Kudos @Sc00bz

JtR uses "skip" when resuming so this was most welcome.

Here are heavier examples: Before:

$ time pp64 <rockyou.lst --skip $((1<<32)) | head -1
ehd8694

real    1m2.629s
user    1m1.803s
sys     0m0.811s

$ time pp64 <rockyou.lst --skip $((1<<33)) | head -1
kgitbB

real    2m0.408s
user    1m59.581s
sys     0m0.821s

The above indicates that 2^34 would take 4 minutes and 2^37 would take half an hour. I do not even want to calculate how long 2^64 would take...

But here's the latest version:

$ time pp64 <rockyou.lst --skip $((1<<33)) | head -1
9313oc.s

real    0m2.074s
user    0m1.916s
sys     0m0.156s

$ time pp64 <rockyou.lst --skip $((1<<62)) | head -1
faLLk27sfhso

real    0m2.066s
user    0m1.903s
sys     0m0.163s

$ time pp64 <rockyou.lst --skip `bc <<< 2^64` | head -1
cgtqVONDnbmo

real    0m2.070s
user    0m1.911s
sys     0m0.160s

It seems to be totally immune against large numbers :smile:

[Sc00bz]

Oh nice now it's fast. Also that was Atom: https://github.com/jsteube/princeprocessor/commit/2b1082d3e42ebecefae2f95d986f162db0b86c3d

[magnumripper]

Yeah kudos to both of you of course. After the latest bug fix it's not that immune to large figures but it's still fast.

$ time pp64 <rockyou.lst --skip `bc <<< 2^40` | head -1
Yoylmt7595

real    0m3.413s
user    0m3.251s
sys     0m0.163s

$ time pp64 <rockyou.lst --skip `bc <<< 2^41` | head -1
7QB8y4s

real    0m4.895s
user    0m4.719s
sys     0m0.175s

$ time pp64 <rockyou.lst --skip `bc <<< 2^42` | head -1
rnLP589720

real    0m8.012s
user    0m7.844s
sys     0m0.169s

[magnumripper]

Here's that last figure running without GMP though (using gcc's __uint128_t)

$ time pp64_u <rockyou.lst --skip `bc <<< 2^42` | head -1
rnLP589720

real    0m3.157s
user    0m2.996s
sys     0m0.160s

More than twice as fast.

$ time pp64_u <rockyou.lst --skip `bc <<< 2^43` | head -1
mpxa1dera

real    0m4.491s
user    0m4.324s
sys     0m0.168s

$ time pp64_u <rockyou.lst --skip `bc <<< 2^44` | head -1
QoZko5t2

real    0m7.049s
user    0m6.882s
sys     0m0.170s

$ time pp64_u <rockyou.lst --skip `bc <<< 2^45` | head -1
bB_qoRll

real    0m12.277s
user    0m12.108s
sys     0m0.169s

[Sc00bz]

This is now fixed in https://github.com/jsteube/princeprocessor/pull/12

[jsteube]

This is awesome!

[magnumripper]

:+1:

$ pp64 <rockyou.lst --keyspace 
1584345389899053977143859

$ time pp64 <rockyou.lst --skip 1584345389899053977143850
R...etniq.etniq.
_...etniq.etniq.
$...etniq.etniq.
!...etniq.etniq.
?...etniq.etniq.
i...etniq.etniq.
q...etniq.etniq.
+...etniq.etniq.
....etniq.etniq.

real    0m2.247s
user    0m2.079s
sys     0m0.163s