JorritMontijn / zetapy

Python code implementing the ZETA-test
GNU General Public License v3.0
23 stars 9 forks source link

Confidence interval for IFR? #5

Closed ErnestJohnston closed 1 year ago

ErnestJohnston commented 1 year ago

I am using the instantaneous firing rate (IFR) from the dRate.vecRate output. Is there a way to return the confidence interval for the IFR at each time point?

JorritMontijn commented 1 year ago

Hi ErnestJohnston,

Thank you for your question. I looked into this a while back when developing the zeta-test. The short answer is no. The long answer is "no, but...".

The IFR does not have a confidence interval, because there is nothing to compute the uncertainty over. The theoretical framework of ZETA is to squash all trials, so the concept of trial-by-trial variability no longer exists. You could compute a confidence interval of your estimate if you select a correct generative statistical model (for example, a renewal process), but that comes with a lot of drawbacks that include having to select a model and fit the parameters, so any estimate would be prone to multiple biases of unknown size.

However, what is possible, is to to compute IFRs for all jitter permutations, which would allow the calculation of a p-value that the IFR is above chance under the null model that the cell is not modulated by your events. However, this approach is 1) very computationally intensive, and 2) is statistically less sensitive than the plain zeta-test. That's why I didn't include this as an option in the toolbox: it would probably take several minutes per cell (assuming a 2-hour recording and an average firing rate of a couple Hz), so this is not something you'd want to do routinely.

That said, you could (fairly) easily program this procedure yourself. If you first run the zeta-test, you get the matrix of random deviation values matRandD (dZeta['matRandD']). If you then use msd.getMultiScaleDeriv() to calculate the IFR for each random deviation vector, this would give you a null-hypothesis IFR distribution at each spike time. You can compare these jittered, random IFRs at each spike time to your original IFR and compute a p-value using its quantile position. If this is the procedure you'd like to follow, but can't figure it out, let me know and I'll see if I can help.

Best, Jorrit

ErnestJohnston commented 1 year ago

Hi Jorrit,

Okay that makes sense. I will forgo this for now, but your explanation has been very helpful. Thank you!