check_db slower than generating the db?

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Create a large database (500M passwords) and add an essid
2. Hard-reset computer while batch-processing
3. run pyrit check_db

What is the expected output? What do you see instead?
I would expect the checking of the results to be quicker than the generation of 
the results, but it seems to be taking at *least* as long as it took to 
actually generate the database.  Some sort of warning about the futility of 
check_db, or at least some sort of progress counter would be nice.

What version of the product are you using? On what operating system?
4.0 w/ CUDA support on Debian sid

Please provide any additional information below.
I noticed that elsewhere someone mentioned that check_db is bound to one thread.

Original issue reported on code.google.com by JeremySa...@gmail.com on 7 Mar 2011 at 9:07

[GoogleCodeExporter]

This is most probably due to the fact that none of the filesystem-blocks are in 
cache after a reboot. What storage-provider do you use?

Original comment by lukas.l...@gmail.com on 9 Mar 2011 at 7:33

• Changed state: WontFix

[GoogleCodeExporter]

This is entirely a local operation.  I am running it on my local computer: a 
6GB/s 7200 RPM 1TB hard drive.  When I say it is "taking a long time" I  mean 
it is taking 9+ hours, and it still hasn't finished checking the first essid, 
whereas it took about 8 hours to batch process an essid with just this machine. 
 I'm not sure how this could be a caching problem.  I would think that the 
files would be properly cached after 8 hours of operation...

Original comment by JeremySa...@gmail.com on 9 Mar 2011 at 5:24

[GoogleCodeExporter]

So you are using the file:// storage?

Original comment by lukas.l...@gmail.com on 10 Mar 2011 at 5:13

• Changed state: Accepted

[GoogleCodeExporter]

yes, correct.

Original comment by JeremySa...@gmail.com on 10 Mar 2011 at 5:48

[GoogleCodeExporter]

I suspect the database to be hundrets of gigabytes in size? I'll write some 
testcase for that...

Original comment by lukas.l...@gmail.com on 10 Mar 2011 at 5:54

[GoogleCodeExporter]

Well, it's not hundreds of GB in size, but it is 61GB.

Original comment by JeremySa...@gmail.com on 10 Mar 2011 at 6:05

[GoogleCodeExporter]

Your workunit_size is at default?

Original comment by lukas.l...@gmail.com on 10 Mar 2011 at 6:06

[GoogleCodeExporter]

I haven't changed any of the options except limit_ncpus.

Original comment by JeremySa...@gmail.com on 10 Mar 2011 at 6:08

[GoogleCodeExporter]

I'll make some significant performance improvements with the next commit

Original comment by lukas.l...@gmail.com on 13 Mar 2011 at 7:15

[GoogleCodeExporter]

Thank you very much!

Original comment by JeremySa...@gmail.com on 13 Mar 2011 at 9:55

[GoogleCodeExporter]

SVN r299 should bring relief

Original comment by lukas.l...@gmail.com on 23 Mar 2011 at 10:45

• Changed state: Fixed

[GoogleCodeExporter]

I have the same problem. After generating db (2 essids) with a wordlist of 
4.0GB, pyrit check_db is running for more than 24 hrs and it's still on the 
first essid.
(version 0.4.0, x86_64 debian squeeze, connected to file://).
However top reports high cpu usage so I guess it's still doing its job:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                           
11732 user1  20   0  126m  45m 6216 R  100  0.6   1959:41 pyrit                 

Should I interrupt it or it could be in an endless loop?

Original comment by terzi.fe...@gmail.com on 28 Jun 2011 at 9:48

[GoogleCodeExporter]

Please update to 0.4.1-dev r299 or higher (== recent svn version). The bug is 
fixed there.

Original comment by lukas.l...@gmail.com on 29 Jun 2011 at 3:34