Index Error on Classify with different probThreshold

[stsmall]

Hi @andrewkern, @dschride, I am getting an odd and intermittent error on Classify.

Traceback (most recent call last): File "classifyChromosome.py", line 64, in writePreds(predictions, probs, coords, outFileName) File "classifyChromosome.py", line 31, in writePreds outLine = coords[i] + [predictions[i]] + list(probs[i]) IndexError: index 4981 is out of bounds for axis 0 with size 4981

Previously with this error I have rebuilt the TrainingSet and retrained and it completes without error. Recently I rebuilt the classifier 3 times and cant seem to classify without errors. Oddly when I change the probThreshold to a lower value, e.g., 0.05 as previously I was using 0.10, it finishes without error. Any suggestions? thanks, @stsmall

[stsmall]

Hi @andrewkern, @dschride, I am still having this error during classify. Sometimes I have to retrain the classifier 3 or 4 times before I can run classify without an error. As you can imagine this takes up an entire day, and I am open to trying any suggestions. thanks, @stsmall

edit: Some results are written even with this error, however the classification is incorrect based on the 'sig' cutoff

[andrewkern]

Hello Scott. Ideally to recreate and fix this bug I would need you to somehow send me:

1) Your classifier 2) Your input directory for the real data you wish to classify (or at least one of the files that produces the error)

These files may be a bit large so email won't work. Do you have any other way of sharing them with me (e.g. a Dropbox or web server where you could up load them)?

Alternatively, I will need some help with the debugging. The first step would be to insert the following line of code before the writePreds call on line 64:

print (len(predictions), len(probs), len(coords))

And let me know what these three numbers are. If the second number is the smallest one, then you can add the following line of code above the if statement on line 20:

print(prob)

Then attach a file containing your entire output. Sorry for the inconvenience, and thanks for your help!

Best,

Dan

[stsmall]

Hi Dan, I will send you a link to the requested files through email. print (len(predictions), len(probs), len(coords)) (3048, 3050, 3050) thanks, scott

[andrewkern]

Dan you are logged in as me!!

[dschride]

Or are you logged in as me? Hmm?

[andrewkern]

In Soviet Russia git hubs you

[andrewkern]

@dschride looks like you pushed code-- @stsmall is this issue now fixed on master?

[stsmall]

Yes. That fixed the error. So I will close this specific issue.

Cause: In some cases the probability between the introgression classes was equal. This was correlated with there only being 2 significant digits in some probs. Currently discussing with Dan as to why this would happen. Using 50 k training examples