benjjneb
commented
5 years ago There is information that you can include taxa classified to higher levels (e.g. down to the family, but not to genus) but how to include taxa classified to lower level (to the genus level) without having information about higher levels (family and higher) ?
For the assignTaxonomy training fasta, you can do this by just including placeholders for the higher taxonomic levels (e.g. "unclassified"). This may or may not achieve exaclty what you want in subsequent phyloseq glomming steps, but it shoudl work fine, e.g. your example id line
>d__unclassified;p__unclassified;c__unclassified;o__unclassified;f__unclassified;g__unclassified_JG_F_00019should work with assignTaxonomy (I think).
Also, have you seen this recent preprint on the "autotax" methodology? It seems to be trying to do the same thing you are: https://www.biorxiv.org/content/10.1101/672873v2
jgrzadziel
marcomeola
digitalwright
Hi everyone! Recently I was working on a huge amount of 16S amplicon data from various environments. It is not a surprise that in some (most of...) samples the unclassified taxa (at least at genus level) was dominant. Sometimes it was about 30, sometimes 80% of the entire microbiome.
DADA2 is an excellent tool to process all steps in the bioinformatic analysis of fastq files. To date, I was used to assigning taxonomy with RDP N.B. classifier (included in the DADA2 package).
However, it is commonly known that it is not good enough to process unculturable taxa which are not present in the databases (so it is not good enough for the majority of taxa in my case). Most of that reads are overclassified, while they are not even closely related to any taxa in the database.
So, I have started using idTaxa from DECIPHER package, which is great for that purpose (however it is also prone to some errors and not so perfect, but currently the most reliable). It appeard that using idTaxa, there is even more "dark matter" fraction of microbes in my samples.
And here are the problems I have noticed with any workflow:
1. Further processing (e.g. using phyloseq) should be done for aggregation of the resulted taxonomy/otu table. This step includes taxonomy aggregation based on each taxonomic level name.
When it comes to fully classified taxa, e.g. Bacillus, there is no problem, because every "hit" of Bacillus is equal and these sequences are mostly identical (~99%), and the count table is simply summarized which is ok.
But when it comes to some unclassified reads for example:...
... in most cases, the sequences are completely different (mostly less than 60-80% similarity), however, they "names" match to each other and then each of "unclassified" read is also simply summarised resulting in the final "unclassified" group with summarised abundance. And here we lost a lot of information. Moreover, for that "summarized" taxa, one representative sequence is chosen by a script, making another confusion. In that situation, you can wrongly interpret your results for example:
Here's how it looks now:
2. Another problem: Each time you sequence different samples, the same "dark matter" microbes are found but you lost the information about them every single time. They are just thrown away and you didn't even notice that there is a pattern for example:
3. What I have tried to do with that issue:
I have written a script that modifies the reads processing after the assignment step (very draft and ugly version) in the following steps:
a) before taxonomy aggregation a unique identifier is assigned to each unclassified read b) these reads are clustered based on 99% sequences similarity c) a unique identifier is assigned to each unclassified "cluster" d) the counts of each unclassified reads are aggregated based on the same cluster membership e) the count table of classified reads is merged with the new "unclassified clusters" count table
Now the result table makes sense: I still have 30% (or more) of unclassified reads, but not as one random representative, instead, I have a few hundreds of low-abundance unclassified representatives which reflects the true microbial diversity. Now the statistics are more reliable and e.g. PCA analysis is not disrupted by our limited taxonomy knowledge. These results should be consistent with future calculations when more taxa will be identified.
Before: (sorted by B1, decreasingly)
After: (sorted by B1, decreasingly)
### And here is when finally I'm asking for help I want to create a custom database for example on the basis of RDP database (or GTDB), which would include these unclassified reads with my identifiers. It would help to assign reads from other samples using the continuously updated database to identify the unclassified reads which are present repeatedly in different experiments.
And here I have two problems (depending on whci classifier I want to use: assignTaxonomy from dada2 or idtaxa from decipher):
There is information that you can include taxa classified to higher levels (e.g. down to the family, but not to genus) but how to include taxa classified to lower level (to the genus level) without having information about higher levels (family and higher) ?
A similar problem is connected with creating a custom database using decipher. In that situation, if I replace the higher levels with "unclassified" like this:
The classifier fails, telling me that I have repeated "unclassified" value at different levels.
Then when I remove all higher taxa like this:
or
or simply
The classifier is not failing, but "JG_F_00019" is treated as a domain, not a genus, because it is the first "not empty" value. And classifying makes no sense now.
I would appreciate any help.