benjjneb
commented
8 months ago ”There are differences in the distribution of q scores within a read, however, which would be relevant to DADA2 filtering. AVITI data will have greater q score variance within a read--in AVITI data you would be more likely to find a single low q score in the middle of a high quality read.”
That does make sense assuming it is true. maxEE is calculating the average expected errors in a read. This is different than calculating average quality score, as quality scores are first transformed into their probability of representing an error (P = 10^-(Q/10)), which makes isolated low quality scores much more impactful to a maxEE filter than an average Q-score filter. There was a paper on this approach that gives more reasoning:
Therefore, they recommend we try tweaking maxEE. Does this sound like a good idea? How should we evaluate/validate the results?
We have a huge number of reads to begin with so that’s no problem, and by truncating close to minimum required overlap we get comparable % of reads through vs. MiSeq. But it feels stupid to throw away high quality data just because I don’t completely understand what’s going on.
Throwing away lower quality data is not stupid. At the end of amplicon sequencing, the parameters we are estimating are the relative abundances -- the absolute number of reads that get through the pipeline is not important beyond how it affects the measurement of the relative abundances. So, pushing through more but lower-quality data can backfire, in that while more reads make it to the end, the measurements of the relative abundance get worse because of the increased noise. Of course there is some optimal level which will vary among datasets, but in general trying to maximize reads passing filtration is not that important. More important is to avoid any systematic loss of data (e.g. of longer amplicons due to merging, etc.). With that said, when I look at the qza's that you posted in the Q2 thread, I'm more concerned abotu the large loss at chimera removal, e.g. nearly 50% loss samples SF1_01 and ZYMO. This is beyond what is expected even for high PCR cycle numbers, and is usually (90%+) due to unremoved technical bases (primers most often, but sometimes barcodes/adapters) that have ambiguous nucleotides in them.
mniku
Avi-Til
IrshadUlHaq1
andressamv
I'm processing our first Element AVITI 16S amplicon sequencing data, using dada2 in QIIME2. I'm wondering how to do this optimally, as it appears that it behaves a bit differently from MiSeq data in dada2:
The phred quality stays much higher than in MiSeq usually, but to my surprise, I still need to truncate the reads just as short to get similar % accepted in DADA2. For more details & actual statistics, see this thread in the QIIME2 forums.
I was first wondering if the AVITI phred scores are a tad optimistic, but got a potentially interesting comment from AVITI bioinformatician. They think the scoring should be quite comparable vs MiSeq BUT: ”There are differences in the distribution of q scores within a read, however, which would be relevant to DADA2 filtering. AVITI data will have greater q score variance within a read--in AVITI data you would be more likely to find a single low q score in the middle of a high quality read.”
Therefore, they recommend we try tweaking maxEE. Does this sound like a good idea? How should we evaluate/validate the results?
We have a huge number of reads to begin with so that’s no problem, and by truncating close to minimum required overlap we get comparable % of reads through vs. MiSeq. But it feels stupid to throw away high quality data just because I don’t completely understand what’s going on.
The QIIME2 guys recommended me to open an issue here, because this calls for a deep level understanding of dada2.