`niche_null_model` giving incorrect results for observed index

[ngotelli]

For the MacArthur's warblers data set, the observed value for the pianka index should be 0.5551383. This value is confirmed if you strip out the row names in the first column of the macwarb data frame and run the pianka function directly on the matrix:

z <- as.matrix(macwarb[,-1],nrow=5)
pianka(z)
> [1]   0.5551383

In the printed output on the quickstart page, this is also what was calculated. However, if you now run this through niche_null_model you see the following

 myWarblers <- niche_null_model(macwarb)
summary(myWarblers)

Time Stamp:  Fri Feb 20 21:25:52 2015 
Random Number Seed Saved:  TRUE 
Number of Replications:  1000 
Elapsed Time:  0.4 secs 
Metric:  Pianka 
Algorithm:  ra3 
Observed Index:  0.40673 
Mean Of Simulated Index:  0.38975 
Variance Of Simulated Index:  0.0022413 
Lower 95% (1-tail):  0.32403 
Upper 95% (1-tail):  0.47414 
Lower 95% (2-tail):  0.31484 
Upper 95% (2-tail):  0.4951 
P(Obs <= null) =  0.688 
P(Obs >= null) =  0.312 
P(Obs = null) =  0 
Standardized Effect Size (SES):  0.3587 

Moreover, the Observed Index value changes with each run of the model, which of course it should not be doing. The observed value should be significantly larger than expected by chance, but now it is effectively random. I am getting similar results with ra1, raw, ra4, and with Czekanowski. The simulated distributions look correct. I strongly suspect that the observed index is now being calculated from one of the simulated matrices, rather than from the original data matrix.

[ngotelli]

Similar problems for co-occurrence module: observed index values keep shifting. Also, "cooc" plots do not look correct: all algorithms seem to be holding row and column sums constant (like sim9). Extreme tail inequalities are also on the wrong side when p <= 0.001 and p>= 0.999 for cases in which observed index is more extreme than any of the simulated values.

@emhart let's skype when you are available.

[emhart]

Good catch Nick,

It was a one line bug in the null model engine from when I changed it to accept arbitrary numbers of parameters for metrics and algos. It's fixed now. However your second point I'm less sure about.

> z <- as.matrix(macwarb[,-1],nrow=5)
> pianka(z)
[1] 0.5551383
> warbMod <- niche_null_model(macwarb)
  |=====================================================================================================| 100%
> warbMod$Obs
[1] 0.5551383

[ngotelli]

Wow, that's great! Thanks for the lightning-fast response! I will keep looking at co-occurrence and get back to you.

Best,

Nick

Quoting Edmund Hart <notifications@github.com>:

Good catch Nick,

It was a one line bug in the null model engine from when I changed
it to accept arbitrary numbers of parameters for metrics and algos.
It's fixed now. However your second point I'm less sure about.

— Reply to this email directly or view it on GitHub[1].  

 

---

Nicholas J. Gotelli       Office Phone: 802-656-0450
Department of Biology     Lab Phone: 802-656-0451   
University of Vermont     Fax: 802-656-2914
Burlington, VT 05405      e-mail: ngotelli@uvm.edu
********************************************************
Home Page (with manuscript pdfs):

http://www.uvm.edu/~ngotelli/homepage.html

Musician's Corner (with mp3s):

http://www.uvm.edu/~ngotelli/musicpage/music.html

NEW: EcoSimR (free software for null model analysis):

http://www.uvm.edu/~ngotelli/EcoSim/EcoSim.html

---

Links:

[1] https://github.com/GotelliLab/EcoSimR/issues/42#issuecomment-75354392

[emhart]

Well, I usually put in a few hours of work after Alistair goes down so you just happened to catch me in the middle of working on it :baby: :sleeping: