Using FTD.comm in a null model

[nikhailarum]

Dear Mr. Kothari

I have used your function FTD.comm to successfully calculate functional diversity for a set of communities in my study area. Having done this I wanted to determine whether the functional diversity I observe in each of my communities is more or less than expected when accounting for the species richness of each site. To do this I run null models in the picante package using a trial swap null model to generate 999 random matrices, maintaining the species richness of each community, and the frequency of occurrence for each species. I do this by using the following code;

replicate (999, randomizeMatrix(my.matrix, null.model = "trialswap", iterations = 1000))

The picante package has existing functions which allow for certain metrics to be easily applied to this function and calculated across the randomized communities. However, as the metric which is calculated using your code is recent and independent of the picante package I have attempted to (unsuccessfully) apply this code across my randomized matrices.

One of my attempts goes as follows: FTD.rand <- FTD.comm((randomizeMatrix(spmat, null.model = "trialswap", iterations = 1000)), tdmat) obs.null.output <- cbind(FTD.comm(tdmat, spmat), replicate(999, FTD.rand(spmat)))

However whenever I try this I get Error message: Error in rowSums(spmat) : 'x' must be an array of at least two dimensions

I have tried defining spmat as an array of 999 random matrices, as well as defining spmat as a randomized matrix and then replicating the FTD.comm function 999 times, however both approaches are unsuccessful. Is there a function in the FD package in which this FTD.comm function can be applied to a null model? Do I need to adapt an apply function to use the metric across randomized matrices? Or do I need to fiddle with the actual FTD.comm function itself in order for it to be applicable across the randomized matrices I generate. Any help with this matter would be greatly appreciated, as applying my metrics to a null model are crucial to my study in order to understand the underlying processes which shape the observed community structure. I am also a relatively inexperienced and limited coder with regards to such matters so I apologize in advance if I have bothered you with something trivial or have, conversely, asked too much of you. If you need any more information or indeed more code samples you are more than welcome to use either this forum or my personal email address.

Kind Regards, Nikhail

[ShanKothari]

Dear Nikhail,

I'm sorry to have taken so long to get back to you! I'm in the middle of an intense field season. I've used those picante functions, and it's not immediately clear to me why this doesn't work. It seems like you're trying to define FTD.rand as a new function, but I've never encountered a situation where someone defines a function in that particular way rather than:

FTD.rand<-function(...){ ... }

You shouldn't need to fiddle with the FTD.comm function -- your basic idea is right that you should be able to randomize the matrix and use FTD.comm on the randomized matrices. One simple (though not necessarily efficient) way to do this would be to store each randomization as a list element, then lapply through all the elements.

If you provide a reproducible example, I can likely get to this sooner. It need not be your actual data -- just any made-up/semi-randomly generated numbers would work.

[nikhailarum]

Hi Dr/Mr. Kothari

Thank you for taking the time to read my issue and offering to help Practice_trait_data.txt PRACTICE_COMMUNITY_MATRIX.txt

I have reproduced the error using a small set of made up data (5 species and 5 sites). Here is the code as follows: library(base)

library(FD)

library(picante)

traits <- read.table("Practice_trait_data.txt", header = T, row.names = 1)

tdmat <- gowdis(traits)

my.matrix <- read.table("PRACTICE_COMMUNITY_MATRIX.txt", header = T, row.names = 1)

spmat <- data.matrix(my.matrix)

FTD.rand <- FTD.comm((randomizeMatrix(spmat, null.model = "trialswap", iterations = 1000)), tdmat)

After running the FTD.rand command I encounter the error Error in rowSums(spmat) : 'x' must be an array of at least two dimensions. Please note for this code I have already defined FTD and FTD.comm functions using your code unchanged. Using this dataset and your code I was able to calculate FTD for the community. It was getting the code to calculate FTD for the randomized communities which I have failed to do and hence I have only included these steps. After running the FTD.rand command I would like to use the; obs.null.output <- cbind(FTD.comm(tdmat, spmat), replicate(999, FTD.rand(spmat))) code to apply this to 999 different matrices. Naturally I would have to get the FTD.rand function to work first. I will try your suggestion of FTD.rand <- function(...){...} however at present I am not entirely certain how this code will have to be laid out. I have attached the made up data for you to use.

Thank you for your kind assistance. Nikhail

[nikhailarum]

Hi Dr/Mr. Kothari

I reproduce the same error using the following code (and the same dataset attached in my previous message):

library(base) library(FD) library(picante) traits <- read.table("Practice_trait_data.txt", header = T, row.names = 1) tdmat <- gowdis(traits) my.matrix <- read.table("PRACTICE_COMMUNITY_MATRIX.txt", header = T, row.names = 1) spmat <- data.matrix(my.matrix) FTD.null <- function(tdmat, spmat) { FTD.comm((randomizeMatrix(spmat, null.model = "trialswap", iterations = 1000)), tdmat) } obs.null.output <- cbind(FTD.comm(tdmat,spmat), replicate(999, FTD.null(tdmat)))

After running the obs.null.output line I receive the error: Error in rowSums(spmat) : 'x' must be an array of at least two dimensions.

Once again, thank you for your time and help. Regards Nikhail

[ShanKothari]

Dear Nikhail,

Your input as an argument to FTD.comm can't be the output of another function. Also, FTD.comm outputs not a data frame containing the FD information about each community, but rather a list including that data frame as well as various parameters summarizing mean information about the set of communities, so cbind won't work quite so simply. I recommend lapply statements, like:

spmat.rand.list<-lapply(1:999,function(x) return(randomizeMatrix(spmat,null.model="trialswap",iterations=1000)))
FTD.rand <- lapply(spmat.rand.list,function(x) return(FTD.comm(spmat=x,tdmat = tdmat)))

You may then want to use further lapply commands to pick out the first object in each sublist of FTD.rand and compile them in a way you can analyze.

It's likely that other 'apply'-style commands, or a for-loop, could get you to the same place, if you don't like lists. But the key thing is you can't pass a function call as an argument in this context.

-Shan

[nikhailarum]

Dear Mr. Kothari

Thank you very much. I have tried this code and it has proved successful. You have been very kind and generous with your time and help. I will make certain to include you in the acknowledgements of my thesis.

Regards Nikhail

[ShanKothari]

No worries! Feel free to pass on further questions (for example, about interpreting the output). Best of luck with your thesis!

-Shan

[nikhailarum]

Dear Mr. Kothari

One last question if I may. I can extract the values I want from each list element, for example extracting my functional trait dispersion from the com.FTD dataframe using FTD.rand[[1]][["com.FTD"]][["qDTM]]. What I want to do is compare my observed qDTM to the 999 values of qDTM for each site at my matrix. Ideally if I could extract all qDTM values from all 999 elements of the list and write a csv of text file I could then just see where my observed value falls within the distribution of these values by simply ranking them. Would you know a way to atleast extract qDTM values from all 999 list elements using an lapply command as then I could just copy and paste into a text file and transfer into excel? I have analysed effectively 50 different communities so extracting qDTM values one list element on each object at a time is not really feasible.

Thank you very much for your help on this

Regards Nikhail

[ShanKothari]

Dear Nikhail,

Imagine output is your output as a list with 1000 elements -- one is the original qDTM output, 999 is the output from the 999 randomizations. Try something like:

qDTM.list<-lapply(output,function(df) df$qDTM)

Now you have a list of vectors which you can turn into a data frame as described here: https://www.r-bloggers.com/converting-a-list-to-a-data-frame/

This should make it easy enough to compare the original qDTM to the qDTM on randomized communities in R.

--Shan

[nikhailarum]

Dear Mr. Kothari

Thank you so much for your assistance throughout our interactions. I am sorry if I have been a bother but you have been an immense help (I am a novice when it comes to R coding). All the best for your future endeavours.

Reagrds Nikhail

On Tue, Sep 11, 2018 at 8:39 PM Shan Kothari notifications@github.com wrote:

Dear Nikhail,

Imagine output is your output as a list with 1000 elements -- one is the original qDTM output, 999 is the output from the 999 randomizations. Try something like:

qDTM.list<-lapply(output,function(df) df$qDTM)

Now you have a list of vectors which you can turn into a data frame as described here: https://www.r-bloggers.com/converting-a-list-to-a-data-frame/

This should make it easy enough to compare the original qDTM to the qDTM on randomized communities in R.

--Shan

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ShanKothari/DecomposingFD/issues/8#issuecomment-420377481, or mute the thread https://github.com/notifications/unsubscribe-auth/Ah4n6tc7TEDBxmuXtUduBGHjNYn_2PGFks5uaANqgaJpZM4UhTIU .

-- Nikhail Arumoogum MSc Candidate School of Life Sciences University of KwaZulu-Natal

[ShanKothari]

Not a problem. Did my suggestion above work? Try it out and make sure that you don't run into any further issues.

[nikhailarum]

Dear Mr. Kothari

I haven't tried your suggestion just yet. It is relatively late in this part of the world (South Africa). I will try it out tomorrow and let you know if it works out.

Regards Nikhail

On Tue, Sep 11, 2018 at 8:59 PM Shan Kothari notifications@github.com wrote:

Not a problem. Did my suggestion above work? Try it out and make sure that you don't run into any further issues.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ShanKothari/DecomposingFD/issues/8#issuecomment-420384327, or mute the thread https://github.com/notifications/unsubscribe-auth/Ah4n6tCoFrCq4Ijl_QOjaT6dxG8ulDmYks5uaAgSgaJpZM4UhTIU .

-- Nikhail Arumoogum MSc Candidate School of Life Sciences University of KwaZulu-Natal

[nikhailarum]

Hi Mr. Kothari

I have tried your suggestion. My list is named FTD.rand and the dataframe within the list which contains the qDTM values is named com.FTD. Applying it to your suggestion, if I am interpreting it correctly would mean something that looked like:

qDTM.list <- lapply(FTD.rand,function(com.FTD) com.FTD$qDTM)

However, this only returns a list of NULL values. Furthermore if I try to extract a value using a command FTD.rand$qDTM a NULLvalue is returned. However when I input the command FTD.rand[[1]][["com.FTD"]][["qDTM"]] the proper values for qDTM from the first list will be reported (I haven't found a way to extract from more than one list). Despite this I am posting to inform you that this is no longer a major issue, as the package r.list allows me to save my list as a .yaml file which can be opened in excel, allowing me to manipulate and organize the data.

Thank you very much for all your assistance.

Kind Regards, Nikhail

[ShanKothari]

Right, I forgot that each list element is itself a list whose first element is the data frame you need. So, I think you could run

qDTM.list <- lapply(FTD.rand,function(com.FTD) com.FTD[[1]]$qDTM)

In any case, I'm glad you figured out a workaround!