klusta-team / spikedetekt2

10 stars 8 forks source link

Fix masked PCA implementation #15

Closed rossant closed 9 years ago

rossant commented 9 years ago

This PR implements the fix described by this e-mail from @kdharris101:

The problem arises when there are no unmasked spikes on a given channel. In this case, setting the PCs to zero is a makes all features 0 on that channel. Even though all the points are masked, KK still has a problem because it replaces masked features with the noise mean and variance, which are zero.

We should change the way we compute PCs to include regularization. This is the best solution, because it also solves another problem. If there were only 1 or 2 spikes unmasked on a given channel, the PCs would also be bogus. The way we could do this is to compute PCs as the eigenvectors of sigma + lamba * sigma0, where sigma is the covariance matrix of unmasked spikes only, and sigma0 is a regularization matrix.

The best choice would be to make sigma0 be the covariance matrix of all unmasked spikes, whatever channel they were recorded from; that way, it would be a “prior” that if there are very few unmasked spikes from this channel, you assume it will be like all other channels. In this case, lambda could be something like 1/N_spikes, so that it is equivalent to one “virtual spike”.