ShanKothari / DecomposingFD

Code to calculate functional trait diversity as defined by Scheiner et al. (2016), and its decomposition into richness, evenness, and dispersion.
8 stars 5 forks source link

FTD.comm not producing results #5

Closed nikhailarum closed 6 years ago

nikhailarum commented 6 years ago

Hello Mr. Kothari

I have recently tried using the code you developed for calculating functional diversity across multiple communities. In using the FTD.comm function on my datasets I get no outputs. I have specified my trait distance matrix as tdmat and my site by species matrix as spmat with species names as rows. My tdmat object is a distance matrix and spmat is a data matrix. They read correctly into R, however when I run the FTD.comm function I get no outputs or error message, and FTD.comm is just listed as a function.

I am using the unedited code from your github page, which reads as:

`function(tdmat,spmat,q=1,abund=F,match.names=T){

if(abund==F){ spmat[spmat>0]<- 1 spmat<-spmat/rowSums(spmat) }

n.comm<-nrow(spmat) if(match.names==T){ sp.arr<-match(rownames(as.matrix(tdmat)),colnames(spmat)) spmat<-spmat[,sp.arr] }

apply FTD to each community in turn

out<-apply(spmat,1,function(x) unlist(FTD(tdmat=tdmat,weights=x,q=q))) df.out<-data.frame(t(out)) rownames(df.out)<-rownames(spmat)

warning for zero-species communities

if(sum(df.out$nsp==0)>0){ warning("at least one community has no species") }

nsp<-sum(colSums(spmat>0))

calculate mean richness, dispersion, evenness, FTD

u.M<-sum(df.out$nsp*df.out$M)/sum(df.out$nsp) u.nsp<-mean(df.out$nsp)

to do: check if u.nsp is always calculated as arithmetic mean

if(q==1){

geometric mean -- limit of generalized mean as q->1

u.qDT<-prod(df.out$qDT)^(1/n.comm)

} else {

generalized mean with m=1-q

u.qDT<-(sum(df.out$qDT^(1-q))/n.comm)^(1/(1-q))

} u.M.prime<-u.M*u.nsp/(u.nsp-1)

calculate mean FTD and evenness

u.qDTM<-1+u.qDT*u.M u.Et<-u.qDT/u.nsp

list more things

list(com.FTD=df.out,nsp=nsp,u.nsp=u.nsp,u.M=u.M,u.M.prime=u.M.prime,u.Et=u.Et,u.qDT=u.qDT,u.qDTM=u.qDTM) }`

My trait distance matrices and site by species matrices are stored as .txt files, however they read into R without any issues. I understand the trait distance matrices have to have all species present, but does this mean that all species must be present as columns in the same order as the site by species matrix? I ask this as this would make the last column redundant, as well as the first row ie. if a sample has 25 species then the trait distance matrix should have 25 rows and 25 columns? My tdmat has 25 species, but 24 columns and 24 rows. Is this the reason I am getting no outputs? I am struggling to understand why I get no outputs, yet no error messages either. Any help with this issue would be greatly appreciated. If needed I can upload examples of my trait distance matrix and my site by species matrix for you to view if needed.

Regards Nikhail

ShanKothari commented 6 years ago

Hello Nikhail,

Imagine that you are storing the output of the function in some object FTD.dat:

FTD.dat<-FTD.comm(tdmat,spmat)

What do you get when you then run:

FTD.dat

Do you get NULL? Or something else?

I understand the trait distance matrices have to have all species present, but does this mean that all species must be present as columns in the same order as the site by species matrix? I ask this as this would make the last column redundant, as well as the first row ie. if a sample has 25 species then the trait distance matrix should have 25 rows and 25 columns? My tdmat has 25 species, but 24 columns and 24 rows.

That's right, they should be in the same order. How do you have 25 species in only 24 columns and 24 rows? I could imagine having a 25 x 24 matrix, if you removed the diagonal (which should be all 0s).

The important thing is that the columns of spmat should have the same species, in the same order, as both the rows and columns of tdmat. If the issue is that your tdmat has the wrong number of rows and columns to match the community abundance matrix (spmat), you should get a non-conformable arrays error. If you don't get that, there must be another issue (perhaps in addition to this one).

--Shan

nikhailarum commented 6 years ago

Dear Mr. Kothari

Thank you very much. This code did work

Regards Nikhail

ShanKothari commented 6 years ago

Great! I hope you were able to resolve the confusion about tdmat as well. Please let me know if I can do anything else to help.

agusnov commented 5 years ago

Dear Mr. Kothari, I'm using your function FTD.comm to get functional diversity among rodents communities along an elevation gradient. I did this using presence absence data, and It seems to be doing ok. But when I'm using the code with abundance data, I got warning messages Warning messages: 1: In FTD(tdmat = tdmat, weights = x, q = q) : input proportional abundances do not sum to 1; summation to 1 forced

I'm not sure if this is a problem? Can you give me an advise on this issue.

Thanks a lot. Agustina

ShanKothari commented 5 years ago

Dear Agustina,

That warning message implies that the abundances in each row of your data add up to more or less than exactly 1. For example, some plant ecologists collect vegetation cover data such that total cover in a plot can add up to more or less than 100%. The code normalizes the abundances so that abundances do sum to 1. This normalization is recommended because diversity abundances typically take in proportional abundances; the goal is usually not to evaluate whether one community has more or fewer individuals than another. However, if you want to skip the normalization, you can remove the corresponding part of the code (or I can do it for you).

Check the structure of your abundance data and see whether the rows sum to 1 or not. Feel free to get back to me here if you need help.

Best, --Shan

agusnov commented 5 years ago

Thanks a lot for your answer. I'm trying to analyse functional diversity along an elevation gradient and compare if FD is higher/lower than expected by SR. When I run the FTD.comm code just like fun=FTD.comm(tdmat,spmat_abund) it seem to be doing OK, althought if I use the code as fun=FTD.comm(tdmat,spmat_abund,q=1,abund=T) the warning message appears. I'm not quite sure how to fix this. Here I attach my spmat with the raw abundances.

Also I was following your gitHub issues with Nikhail to run the null model. I run this code spmat.rand.list<-lapply(1:999,function(x) return(randomizeMatrix(spmat,null.model="trialswap",iterations=1000))) FTD.rand <- lapply(spmat.rand.list,function(x) return(FTD.comm(tdmat,spmat)))

But It seems to be replicating the same species matrix again and again. Here I attach an example, so it would be easy to find a solution.

traits2-prueba2.xlsx abundx_prueba.xlsx

Here is my code abuL<- read.csv2(file="abundx_prueba.csv", header=TRUE, sep=",") names(abuL) abuL1=abuL[1:20,2:6] spmat_abund=as.matrix(abuL1)

tramat<- read.csv2(file="traits2-prueba1.csv", header=TRUE, sep=",") tra=tramat[,2:18] rownames(tra)<-(tramat$Species)

Ordinal variables "Weight..g. HB..mm. Tail.length Hind.foot Ear.length

tr <- tramat[, 11:15] rownames(tr)<-(tramat$Species) tr1 <- log(tr + 1) # Quantitative variables rownames(tr)<-(tramat$Species)

Variables multichoice

tra1 <- tramat[, 2:4] rownames(tra1)<-(tramat$Species) tra2 <- tramat[, 5:6] rownames(tra2)<-(tramat$Species) tra3 <- tramat[, 7:10] rownames(tra3)<-(tramat$Species)

returns a data frame with the following attributes: col.blocks specifies the number of

columns per fuzzy variable; col.num specifies which variable each column belongs to

trs1 <- prep.binary(tra1, col.blocks = 3, label = "diet") trs2 <- prep.binary(tra2, col.blocks = 2, label = "Habits") trs3 <- prep.binary(tra3, col.blocks = 4, label = "locomotion")

making a list of all traits

klist <- ktab.list.df(list(tr, trs1, trs2, trs3)) klist

Calculation of the global functional distances between species

gdist <- dist.ktab(klist, type= c("Q", "B","B","B"))
summary(gdist) gdist is.euclid(gdist) #no missing data in the matrix tdmat=as.matrix(gdist) tdmat

FUNCTIONAL DIVERSITY fun<-FTD.comm(tdmat,spmat_abund) fun

fun1=FTD.comm(tdmat,spmat_abund,q=1,abund=T) fun1

Null model

library(FD) library(picante)

spmat.rand.list<-lapply(1:5,function(x) return(randomizeMatrix(spmat_abund,null.model="trialswap",iterations=1000))) FTD.rand <- lapply(spmat.rand.list,function(x) return(FTD.comm(tdmat,spmat_abund)))

qDTM.list <- lapply(FTD.rand,function(com.FTD) com.FTD[[1]]$qDTM)

Thanks a lot for your help.

ShanKothari commented 5 years ago

Hi Agustina,

It may take me some time to work through your example. Feel free to bug me in a couple days if I haven't responded.

Best, --Shan

agusnov commented 5 years ago

Dear Shan, I just want to remind you (or as you said bug you) with my FTD.comm question. I really appologise for this, but I cant find the solution myself. Thanks a lot Agustina

El jue., 22 ago. 2019 a las 2:06, Shan Kothari (notifications@github.com) escribió:

Hi Agustina,

It may take me some time to work through your example. Feel free to bug me in a couple days if I haven't responded.

Best, --Shan

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ShanKothari/DecomposingFD/issues/5?email_source=notifications&email_token=AHXVGGQTSQEVCUEZVREGFT3QFYNEJA5CNFSM4EZREVAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD434QXQ#issuecomment-523749470, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXVGGWKZUM7CX324UHI4FTQFYNEJANCNFSM4EZREVAA .

-- Agustina Novillo anovillo@ibn-conicet.gob.ar Instituto de Biodiversidad Neotropical Horco Molle s/n, Yerba Buena, Tucumán +54(381)4859184

ShanKothari commented 4 years ago

Hi Agustina,

Two notes:

  1. Yes, your abundance data are being normalized by force to fit the assumptions of the metric. That's not a problem!
  2. Your problem with randomization lies in the following lines:
spmat.rand.list<-lapply(1:5,function(x) return(randomizeMatrix(spmat_abund,null.model="trialswap",iterations=1000)))
FTD.rand <- lapply(spmat.rand.list,function(x) return(FTD.comm(tdmat,spmat_abund)))

The first line is randomizing the data. But the second line isn't actually applying the FTD.comm function to the randomized data. It just applies it to the original data over and over! Can you think about how to fix this? I can give you an answer, but it might be better if I gave you some time to work out a solution.

Finally: When you get to the point of comparing against the randomization-based null distribution, make sure to compare abundance-weighted qDTM to other abundance-weighted values, and non-weighted values to other non-weighted values!

Best, --Shan

ShanKothari commented 4 years ago

Hi Agustina,

Did my advice above resolve your problem?

--Shan

agusnov commented 4 years ago

Dear Shan, thanks for your concern. I looked to that code several times, and after a while I think, I figured out what was going on. I changed the FTD.rand function to: FTD.rand <- lapply(spmat.rand.list,function(x) return(FTD.comm(spmat=x,tdmat = tdmat))) and now it seems to be working.

Thanks a lot for your help

El mar., 17 sept. 2019 a las 0:32, Shan Kothari (notifications@github.com) escribió:

Hi Agustina,

Did my advice above resolve your problem?

--Shan

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ShanKothari/DecomposingFD/issues/5?email_source=notifications&email_token=AHXVGGVIHM7HJ4INOEGZQRLQKBFWTA5CNFSM4EZREVAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD63FQVA#issuecomment-532043860, or mute the thread https://github.com/notifications/unsubscribe-auth/AHXVGGW4TOS3OEZE4VHHPS3QKBFWTANCNFSM4EZREVAA .

-- Agustina Novillo anovillo@ibn-conicet.gob.ar Instituto de Biodiversidad Neotropical Horco Molle s/n, Yerba Buena, Tucumán +54(381)4859184

ShanKothari commented 4 years ago

Yes, I think that's the solution. Glad to hear it worked out! Feel free to ask any other questions you might have.