search

biocore / emp

Code repository of the Earth Microbiome Project.
http://www.earthmicrobiome.org
BSD 3-Clause "New" or "Revised" License
154 stars 73 forks source link

blooming taxa analysis #39

Open cuttlefishh opened 8 years ago

cuttlefishh commented 8 years ago

@rob-knight suggested: we should definitely do some blooming taxa analysis as per the Shade paper and maybe encourage Ashley to do that.

Ashley Shade is at Michigan State.

ashleyshade commented 8 years ago

Sure let me know how I can help

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics On Dec 10, 2015 7:41 PM, "Luke Thompson" notifications@github.com wrote:

@rob-knight https://github.com/rob-knight suggested: we should definitely do some blooming taxa analysis as per the Shade paper and maybe encourage Ashley to do that

Ashley Shade is at Michigan State

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39.

cuttlefishh commented 8 years ago

Thanks, Ashley! Can you send a link to the manuscript Rob is referring to?

Luke

On Dec 10, 2015, at 5:13 PM, Ashley Shade notifications@github.com wrote:

Sure let me know how I can help

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics On Dec 10, 2015 7:41 PM, "Luke Thompson" notifications@github.com wrote:

@rob-knight https://github.com/rob-knight suggested: we should definitely do some blooming taxa analysis as per the Shade paper and maybe encourage Ashley to do that

Ashley Shade is at Michigan State

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39.

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-163801614.

rob-knight commented 8 years ago

A meta-analysis of changes in bacterial and archaeal communities with time. http://www.ncbi.nlm.nih.gov/pubmed/23575374 Conditionally rare taxa disproportionately contribute to temporal changes in microbial diversity. http://www.ncbi.nlm.nih.gov/pubmed/25028427

On Dec 11, 2015, at 10:01 AM, Luke Thompson notifications@github.com wrote:

Thanks, Ashley! Can you send a link to the manuscript Rob is referring to?

Luke

On Dec 10, 2015, at 5:13 PM, Ashley Shade notifications@github.com wrote:

Sure let me know how I can help

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics On Dec 10, 2015 7:41 PM, "Luke Thompson" notifications@github.com wrote:

@rob-knight https://github.com/rob-knight suggested: we should definitely do some blooming taxa analysis as per the Shade paper and maybe encourage Ashley to do that

Ashley Shade is at Michigan State

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39.

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-163801614.

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-164004569.

cuttlefishh commented 8 years ago

Thanks, Rob!

On Dec 11, 2015, at 10:02 AM, Rob Knight notifications@github.com wrote:

A meta-analysis of changes in bacterial and archaeal communities with time. http://www.ncbi.nlm.nih.gov/pubmed/23575374 Conditionally rare taxa disproportionately contribute to temporal changes in microbial diversity. http://www.ncbi.nlm.nih.gov/pubmed/25028427

On Dec 11, 2015, at 10:01 AM, Luke Thompson notifications@github.com wrote:

Thanks, Ashley! Can you send a link to the manuscript Rob is referring to?

Luke

On Dec 10, 2015, at 5:13 PM, Ashley Shade notifications@github.com wrote:

Sure let me know how I can help

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics On Dec 10, 2015 7:41 PM, "Luke Thompson" notifications@github.com wrote:

@rob-knight https://github.com/rob-knight suggested: we should definitely do some blooming taxa analysis as per the Shade paper and maybe encourage Ashley to do that

Ashley Shade is at Michigan State

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39.

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-163801614.

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-164004569.

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-164004840.

ashleyshade commented 8 years ago

http://mbio.asm.org/content/5/4/e01371-14 http://www.cell.com/trends/microbiology/abstract/S0966-842X%2815%2900018-9

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics Biomedical Physical Sciences rm 6179 517-884-5399 tel:517-884-5399 ShadeLab http://ashley17061.wix.com/shadelab

On Dec 11, 2015, at 1:01 PM, Luke Thompson notifications@github.com wrote:

Thanks, Ashley! Can you send a link to the manuscript Rob is referring to?

Luke

On Dec 10, 2015, at 5:13 PM, Ashley Shade notifications@github.com wrote:

Sure let me know how I can help

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics On Dec 10, 2015 7:41 PM, "Luke Thompson" notifications@github.com wrote:

@rob-knight https://github.com/rob-knight suggested: we should definitely do some blooming taxa analysis as per the Shade paper and maybe encourage Ashley to do that

Ashley Shade is at Michigan State

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39.

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-163801614.

— Reply to this email directly or view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-164004569.

ashleyshade commented 8 years ago

Just want to reiterate - would love to contribute to this! let me know if and how we can!

cuttlefishh commented 8 years ago

Ashley, we would love to have you help also! We are working toward a paper on 20k samples from the EMP, focusing on those with the best metadata plus a few other factors. We are currently working to identify the final set of 20k samples, and meanwhile putting together a set of 2k samples for testing analysis methods. Most of the samples in the EMP do not have a specific temporal component. Do you have some thoughts for how your work on conditionally rare taxa (CRT) would translate to a dataset without a major temporal component?

ashleyshade commented 8 years ago

The temporal component is key for the "blooming" interpretation. However, the algorithm is generic in that it could easily be applied over space instead of time, but then the interpretation would concern the biogeography of populations that are typically rare but in some localities abundant. In these cases, we would hypothesize that contextual data would explain any local high abundance of otherwise rare taxa. In any one particular habitat, this would be a bit boring (the local environment drives differences in community structure - duh) but with the large EMP dataset it may be interesting to determine macroscale patterns of rarity. Perhaps we could use a model of the biogeographic distributions (based on the whole dataset) to test if these occurrences are different than expected given neutral assumptions.

cuttlefishh commented 8 years ago

Hi @ashleyshade -- we are mobilizing forces to analyze the first 20k EMP samples (28,685 samples with >30,000 reads per sample -> 20,347 samples). How do the next few weeks look for you? Your idea to look at macroscale patterns of rarity sounds promising. Is there a version of the BIOM table that you'd prefer to work with? We have closed-reference (Greengenes and Silva), open-reference (Greengenes), "deblurred" actual sequences, and hope to also have de novo (Swarm).

cuttlefishh commented 8 years ago

One clarification: The sequences per sample will vary depending on the OTU picking method. The 30k seqs/sample is for the closed-ref GG table. We are comparing GG to Silva, and also looking at the other OTU tables, to decide on a reasonable set of ~20k samples to include in all analyses.

ashleyshade commented 8 years ago

Hi Luke, Awesome!! I am traveling but send the biom along with the metadata and I'll have a start as I am able.

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics On Jun 16, 2016 21:34, "Luke Thompson" notifications@github.com wrote:

Hi @ashleyshade https://github.com/ashleyshade -- we are mobilizing forces to analyze the first 20k EMP samples (28,685 samples with >30,000 reads per sample -> 20,347 samples). How do the next few weeks look for you? Your idea to look at macroscale patterns of rarity sounds promising. Is there a version of the BIOM table that you'd prefer to work with? We have closed-reference (Greengenes and Silva), open-reference (Greengenes), "deblurred" actual sequences, and hope to also have de novo (Swarm).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-226589893, or mute the thread https://github.com/notifications/unsubscribe/ACLJlW5hXdF22dz-Q5Puzmqz0kBGCs8rks5qMaU2gaJpZM4GzOX2 .

ashleyshade commented 8 years ago

O yeah and I think closed reference is best for this.

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics On Jun 17, 2016 12:15, "Ashley Shade" shade.ashley@gmail.com wrote:

Hi Luke, Awesome!! I am traveling but send the biom along with the metadata and I'll have a start as I am able.

Ashley Shade, Ph.D. Michigan State University Microbiology and Molecular Genetics On Jun 16, 2016 21:34, "Luke Thompson" notifications@github.com wrote:

Hi @ashleyshade https://github.com/ashleyshade -- we are mobilizing forces to analyze the first 20k EMP samples (28,685 samples with >30,000 reads per sample -> 20,347 samples). How do the next few weeks look for you? Your idea to look at macroscale patterns of rarity sounds promising. Is there a version of the BIOM table that you'd prefer to work with? We have closed-reference (Greengenes and Silva), open-reference (Greengenes), "deblurred" actual sequences, and hope to also have de novo (Swarm).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/biocore/emp/issues/39#issuecomment-226589893, or mute the thread https://github.com/notifications/unsubscribe/ACLJlW5hXdF22dz-Q5Puzmqz0kBGCs8rks5qMaU2gaJpZM4GzOX2 .

cuttlefishh commented 8 years ago

OK great! We will get the closed-reference OTU table on the FTP site for you (probably Silva bc it assigns more reads than GG). I will email a condensed version of the mapping file, and you can let me know if it has what you need (the full one is 117MB or ~6x larger than this one).

cuttlefishh commented 8 years ago

Ashley has posed these questions:

  1. Are there taxa that are universally rare? If so, who are they and is there anything known about their biology that informs their ecology (e.g. spore formers/known dormant states)? Is their rarity an artifact of detection?
  2. Are there taxa that are consistently rare in some biomes but prevalent in others (suggesting connectivity/seeding between habitats as observed for aquatic systems by terrestrial systems by Del Giorgio’s group recently)
  3. Is there phylogenetic signal among rare taxa, within a biome? Is the composition of rare taxa within a biome different than the composition of the prevalent taxa?
  4. CRT - can we detect “blooming” populations over space or time?
cuttlefishh commented 7 years ago

I have passed to @ashleyshade the closed-ref table with 5000 samples plus mapping file of those samples only. The files are on my Dropbox: https://www.dropbox.com/sh/pd5zjrfb0i7li6r/AAA1Wj7fnEQPucAWxE9QdDE-a?dl=0