Closed GoogleCodeExporter closed 9 years ago
I strongly suspect this to be a problem with your hardware. The fact that
'selftest'
completes without error (e.g. CUDA_ERROR_LAUNCH_FAILED) but reports bad results
points to that.
Try removing one of the GTX295 or one of your RAM-chips from your setup...
Original comment by lukas.l...@gmail.com
on 20 Oct 2009 at 6:33
Many thanks for your response – I’m sure you’re right.
Both RAM and cards were tested several days ago, to ensure they weren’t
defective.
RAM passed a 90 minute memtest and the cards performed to expectations when
running
a Windows-based CUDA application (EWSA from Elcomsoft).
Removing a RAM module hasn’t made any difference; and removing a card
doesn’t seem
warranted, given their excellent EWSA and (intermittent) pyrit results.
Could the problem lie in the cards’ new architecture ?
They are both Gigabyte rev 2.0 versions of the GeForce GTX 295 and the 2
GPU’s on
each card share a single PCB, whereas the original version had 2 PCB’s, one
for each
GPU. Gigabyte designates them as GV-N295-181-B rev 2.0 cards.
Original comment by fl...@xtra.co.nz
on 21 Oct 2009 at 9:01
A new architecture has no such effect.
I still strongly suspect one of your hardware components to be defective. The
command
'selftest' compares the results that come back from your GPUs to pre-computed
values
hardwired into Pyrit's sourecode. The error message "At least some results..."
means
that there were bit-errors in some of the results which passed the whole
CUDA-driver
stack without being detected. The random crashes (CUDA_ERROR_LAUNCH_FAILED) also
speak for themselves.
You can try the following to try to pinpoint the GPU that causes the problem:
Open /usr/lib/python2.?/site-packages/cpyrit/cpyrit.py and find the line
'for dev_idx, device in enumerate(_cpyrit_cuda.listDevices()):'
modify it like this:
'for dev_idx, device in enumerate(_cpyrit_cuda.listDevices()[0]):'
This will cause Pyrit to only use the first GPU (See list_cores). If you don't
experience any errors (selftest/benchmark/cowpatty), you can increase [0] to
[1] and
re-test.
Original comment by lukas.l...@gmail.com
on 22 Oct 2009 at 10:15
Your last suggestion was very much appreciated for its clarity and promptness.
I’ve worked through the exercise twice, just to be sure a “pattern” I had
noticed,
was being repeated. My procedure was:
1. Edit the cpyrit.py line by adding [0] as suggested, and then shutdown.
2. Restart and run pyrit’s list_cores, selftest, benchmark, and a passthrough
test.
[The selftest, benchmark and passthrough results were all successful]
3. Edit the cyprit.py line by changing [0] to [1] and then shutdown.
4. Restart and run pyrit’s list_cores, selftest, benchmark, and a passthrough
test.
[Again, the selftest, benchmark and passthrough results were successful]
At that stage I noticed the same device (#1 GeForce GTX 295) was being seen by
list_cores, selftest and benchmark.
My original, unedited list_cores output is:
The following cores seem available...
#1: 'CUDA-Device #1 'GeForce GTX 295''
#2: 'CUDA-Device #2 'GeForce GTX 295''
#3: 'CUDA-Device #3 'GeForce GTX 295''
#4: 'CUDA-Device #4 'GeForce GTX 295''
#5: 'CPU-Core (SSE2)'
#6: 'CPU-Core (SSE2)'
#7: 'CPU-Core (SSE2)'
#8: 'CPU-Core (SSE2)'
And an earlier benchmark test listed results for those same cores
The edited [0] list_cores output is:
The following cores seem available...
#1: 'CUDA-Device #1 'GeForce GTX 295''
#2: 'CPU-Core (SSE2)'
#3: 'CPU-Core (SSE2)'
#4: 'CPU-Core (SSE2)'
#5: 'CPU-Core (SSE2)'
#6: 'CPU-Core (SSE2)'
#7: 'CPU-Core (SSE2)'
#8: 'CPU-Core (SSE2)'
Results for selftest and benchmark used the above cores.
The edited [1] list_cores output is:
The following cores seem available...
#1: 'CUDA-Device #1 'GeForce GTX 295''
#2: 'CPU-Core (SSE2)'
#3: 'CPU-Core (SSE2)'
#4: 'CPU-Core (SSE2)'
#5: 'CPU-Core (SSE2)'
#6: 'CPU-Core (SSE2)'
#7: 'CPU-Core (SSE2)'
#8: 'CPU-Core (SSE2)'
Again, the results for selftest and benchmark used the above cores. I
half-expected
to see a different CUDA-Device, maybe the #2 GeForce GTX 295 ?
Experimenting further: list_cores results for an edit [2] and then an edit [3]
also
picked out the same device, namely the #1 'GeForce GTX 295'.
That prompts me to enquire if the cpyrit.py file can be further edited to pick
out
the other GeForce GTX 295 devices #2, #3, and #4 - either in isolation, or
perhaps
as a progressive combination ?
Again - many thanks for your help and patience.
Original comment by fl...@xtra.co.nz
on 24 Oct 2009 at 1:51
Blah my fault. It should read
'for dev_idx, device in enumerate(_cpyrit_cuda.listDevices()[:1]):'
Change the [:1] to [:2], [:3] and finally [:4]
Original comment by lukas.l...@gmail.com
on 24 Oct 2009 at 6:54
Thank you for that – the errant “:” made all the difference.
Apart from the CPU cores:
The [:1] edit revealed: #1: 'CUDA-Device #1 'GeForce GTX 295''
The [:2] edit revealed: #1: 'CUDA-Device #1 'GeForce GTX 295''
#2: 'CUDA-Device #2 'GeForce GTX 295''
The [:3] edit revealed: #1: 'CUDA-Device #1 'GeForce GTX 295''
#2: 'CUDA-Device #2 'GeForce GTX 295''
#3: 'CUDA-Device #3 'GeForce GTX 295''
The [:4] edit revealed: #1: 'CUDA-Device #1 'GeForce GTX 295''
#2: 'CUDA-Device #2 'GeForce GTX 295''
#3: 'CUDA-Device #3 'GeForce GTX 295''
#4: 'CUDA-Device #4 'GeForce GTX 295''
Edits [:1], [:2] and [:3] passed pyrit’s list_cores, selftest, benchmark, and
a
passthrough test.
Edit [:4] completed list_cores OK – completed the selftest with
“WARNING!”, and
wouldn’t complete the benchmark or passthrough tests.
So GPU #4 has been flagged with a defect that I’ll have to get sorted, but in
the
interim I can usefully use that edit [:3] option, as the performance of 3
devices is
still quite respectable.
Again – let me repeat my appreciation for your considerable help and
patience. This
must rate as one of the more-lengthy Issues (now Fixed) occupying space at
Google
Code. Is there some practical way I can compensate for this, so that your very
impressive project continues to flourish ?
Original comment by fl...@xtra.co.nz
on 24 Oct 2009 at 11:13
You should run 'selftest' multiple times to be sure that the defect is isolated
in
GPU #4...
Closed for now
Original comment by lukas.l...@gmail.com
on 25 Oct 2009 at 6:03
Original issue reported on code.google.com by
fl...@xtra.co.nz
on 20 Oct 2009 at 3:21