Deal with columns of all zeros

[willwerscheid]

In ashr it's possible that we get columns of all zeros when calculating likelihood matrices (and thus -Inf in llik matrices). In the example that's throwing an error for me, the smallest mixture component (in a mixture of uniforms) is [0, 5.80e-05], but the data points x are all around 0.15 with small standard errors, so log_comp_dens_conv.unimix evaluates the likelihood that they come from this component as numerically zero. So we get a column of all zeros in the likelihood matrix. This causes mixsqp to complain, but it should be able to handle such a case.

[pcarbo]

@willwerscheid If a column of the matrix L is entirely zeros, then it should be the case that the mixture component associated with that column should also be zero. Correct?

[willwerscheid]

@pcarbo Do you mean that the estimated pi for that component should be zero? If so, then yes, correct.

[willwerscheid]

Here's a quick example that reproduces the error: ashr::ash(.015 + .00008 * rnorm(40), rep(.00008, 40), mixcompdist="+uniform", method="shrink", optmethod="mixSQP")

[pcarbo]

@willwerscheid I am a bit hesistant to add this feature to mixsqp---the user should take some effort in preparing the data to accomodate mixsqp---but I will relent in this case.

[pcarbo]

@willwerscheid Just to clarify, I think you mean columns of all zeros, not -Inf. (All entries in the matrix L must be non-negative.)

[willwerscheid]

@pcarbo I have no skin in this game. You can hash it out with @stephens999 . But one of mixsqp or ashr needs to be fixed: https://github.com/stephens999/ashr/issues/89

[stephens999]

Yes, it will be 0s in L. -inf in log L but mixsqp does not have to worry about that.

Matthew

On Nov 21, 2018 16:03, "Peter Carbonetto" notifications@github.com wrote:

@willwerscheid https://github.com/willwerscheid Just to clarify, I think you mean columns of all zeros, not -Inf. (All entries in the matrix L must be non-negative.)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stephenslab/mixsqp/issues/18#issuecomment-440824591, or mute the thread https://github.com/notifications/unsubscribe-auth/ABt4xbH67Zvkv64SVVB4am0cmONRYqlFks5uxc2ogaJpZM4Yt5nT .

[pcarbo]

@willwerscheid Looks like you are addressing this in ashr, which I think is the right approach after all. (As you observed, this will be an issue for multiple optimization methods.) So I will close this here.

[stephens999]

I personally think that mixsqp should deal with this. It is not that out of bounds, and the solution is clear.

[pcarbo]

@willwerscheid Please check that this works for you now. I'm closing this issue.