Open emankhalaf opened 10 months ago
This is a new one for me. Can you tell me more?
This warning did not come up in learnErrors
?
What was the exact dada
call that produced the warning?
What is the size of the input data? E.g. reads/lengths.
@benjjneb
Thank you for your reply.
This warning did not come up in learnErrors?
No.
What was the exact dada call that produced the warning?
Please see below:
dd <- dada(filts, err=err, pool="pseudo", BAND_SIZE=32, OMEGA_A=1e-10, DETECT_SINGLETONS=FALSE, multithread=TRUE)
What is the size of the input data? E.g. reads/lengths.
50 GB, approx. 1020 samples. I am following the DADA2 tutorial for PacBio reads. I will copy these codes here:
filts <- file.path("home/Documents/sequences", "noprimers", "filtered", basename(fns))
track <- filterAndTrim(nops, filts, minQ=3, minLen=1300, maxLen=1600, maxN=0, rm.phix=FALSE, maxEE=2, verbose=TRUE)
err <- learnErrors(filts, errorEstimationFunction=PacBioErrfun, BAND_SIZE=32, multithread=TRUE)
Now, I exported the feature table, tax table, sequences, alignment, and the tree as rds, and I exported the phyloseq object as well. The sequences used to construct the tree are 6823 sequences.
Are there any concerns regarding the warning that I posted above from dada step?
I hope I have described the issue clearly. Thanks!
Now, I exported the feature table, tax table, sequences, alignment, and the tree as rds, and I exported the phyloseq object as well. The sequences used to construct the tree are 6823 sequences.
So after the warning, you still are getting what looks like valid output?
Are there any concerns regarding the warning that I posted above from dada step?
I am a bit concerned, as integer overflows on the R side (which I'm guessing is what is sparking this error) could lead to incorrect output. What is the size of the sequence table you are getting after denoising here?
@benjjneb 23.8 MB is the size of the sequence table output from the denoising step. Is it reasonable for this step?
By size, I mean dim(seqtab)
. And maybe also summary(nchar(getSequences(seqtab)))
@benjjneb
dim(seqtab)
is 1019 6907 (1019 samples having 6907 sequences)
summary(nchar(getSequences(seqtab)))
Min. 1st Qu. Median Mean 3rd Qu. Max.
1302 1412 1437 1434 1462 1596
Hmm, that doesn't seem to be a problematic size.
One guess is that there could be an overflow happening in the pseudo-pooling code. Would you be able to test that by running the dada(...)
command again that produced the warning, but with pool=FALSE
, to see if the warning goes away?
Hi @benjjneb
I am processing over 1000 16S PacBio SMRT sequenced samples using DADA2 workflow in R, and I got this warning message from dada step using pool="pseudo" :
"Warning: NAs produced by integer overflowWarning: NAs produced by integer overflowWarning: NAs produced by integer overflow selfConsist step 2Warning: NAs produced by integer overflowWarning: NAs produced by integer overflowWarning: NAs produced by integer overflow"
However, the denoise step is finished successfully. So, my question is, is there anything to do with this warning message or just proceed in the workflow? I think this warning concerns R's capacity to denoise such a large number of samples, correct?
Thanks in advance!
Eman