filterAndTrim: No reads passed the filter

[sfeds]

Hi, I've been using DADA2 for a while, and never had this, but lately I've had this with the last runs. Version:

library(dada2); packageVersion("dada2") Loading required package: Rcpp [1] ‘1.10.0’

Quality profiles look good

Then comes filterAndTrim, and I used standard settings and get this error:

out <- filterAndTrim(fnFs, filtFs, fnRs, filtRs, truncLen=c(280,200), maxN=0, maxEE=c(2,2), truncQ=2, rm.phix=TRUE, compress=TRUE, multithread=TRUE) # On Windows set multithread=FALSE

Warning message: In filterAndTrim(fnFs, filtFs, fnRs, filtRs, truncLen = c(280, 200), : No reads passed the filter. Please revisit your filtering parameters.

Any tips? Highly appreciated!

[benjjneb]

The early drop in the median quality scores is unusual. Sometimes Illumina runs have funny things that happen early in the sequencing, perhaps due to calibration issues with the base-calling software.

Is the warning of no reads passing the filter happening for every sample?

If so, I'd recommend trying to narrow down what parameter is causing the problem by turning off all filters but one, until finding the one that caused the problem. e.g.

filterAndTrim(fnFs[[1]], filtFs[[1]], fnRs[[1]], filtRs[[1]], truncLen=c(280,200), 
maxN=0, truncQ=0, rm.phix=TRUE,  # turn off truncQ and rm.phix filtering
verbose=TRUE)
# and...
filterAndTrim(fnFs[[1]], filtFs[[1]], fnRs[[1]], filtRs[[1]], truncLen=c(280,200), 
maxN=999, truncQ=2, rm.phix=TRUE,  # turn off maxN and rm.phix filtering
verbose=TRUE)
# and so forth

[sfeds]

Hi,

I've tried switching off every parameter, or increasing thresholds... Unfortunately still get the same message. Let me quickly work you through my data:

knitr::opts_chunk$set(echo = TRUE)
library(dada2); packageVersion("dada2")

[1] ‘1.10.0’

path <- "/s-schijf/fuentess/Data/Program/Run3_16S/Unzipped" 
list.files(path)

[1] "1_S1_L001_R1_001.fastq" "1_S1_L001_R2_001.fastq" "13_S13_L001_R1_001.fastq" [4] "13_S13_L001_R2_001.fastq" "14_S14_L001_R1_001.fastq" "14_S14_L001_R2_001.fastq" [7] "15_S15_L001_R1_001.fastq" "15_S15_L001_R2_001.fastq" "16_S16_L001_R1_001.fastq" [10] "16_S16_L001_R2_001.fastq" "17_S17_L001_R1_001.fastq" "17_S17_L001_R2_001.fastq" [13] "2_S2_L001_R1_001.fastq" "2_S2_L001_R2_001.fastq" "25_S25_L001_R1_001.fastq" [16] "25_S25_L001_R2_001.fastq" "26_S26_L001_R1_001.fastq" "26_S26_L001_R2_001.fastq" [19] "27_S27_L001_R1_001.fastq" "27_S27_L001_R2_001.fastq" "28_S28_L001_R1_001.fastq" [22] "28_S28_L001_R2_001.fastq" "29_S29_L001_R1_001.fastq" "29_S29_L001_R2_001.fastq" [25] "3_S3_L001_R1_001.fastq" "3_S3_L001_R2_001.fastq" "37_S37_L001_R1_001.fastq" [28] "37_S37_L001_R2_001.fastq" "38_S38_L001_R1_001.fastq" "38_S38_L001_R2_001.fastq" [31] "39_S39_L001_R1_001.fastq" "39_S39_L001_R2_001.fastq" "4_S4_L001_R1_001.fastq"
[34] "4_S4_L001_R2_001.fastq" "40_S40_L001_R1_001.fastq" "40_S40_L001_R2_001.fastq" [37] "41_S41_L001_R1_001.fastq" "41_S41_L001_R2_001.fastq" "49_S49_L001_R1_001.fastq" [40] "49_S49_L001_R2_001.fastq" "5_S5_L001_R1_001.fastq" "5_S5_L001_R2_001.fastq"
[43] "50_S50_L001_R1_001.fastq" "50_S50_L001_R2_001.fastq" "51_S51_L001_R1_001.fastq" [46] "51_S51_L001_R2_001.fastq" "52_S52_L001_R1_001.fastq" "52_S52_L001_R2_001.fastq" [49] "53_S53_L001_R1_001.fastq" "53_S53_L001_R2_001.fastq" "61_S61_L001_R1_001.fastq" [52] "61_S61_L001_R2_001.fastq" "62_S62_L001_R1_001.fastq" "62_S62_L001_R2_001.fastq" [55] "63_S63_L001_R1_001.fastq" "63_S63_L001_R2_001.fastq" "64_S64_L001_R1_001.fastq" [58] "64_S64_L001_R2_001.fastq" "65_S65_L001_R1_001.fastq" "65_S65_L001_R2_001.fastq" [61] "73_S73_L001_R1_001.fastq" "73_S73_L001_R2_001.fastq" "74_S74_L001_R1_001.fastq" [64] "74_S74_L001_R2_001.fastq" "75_S75_L001_R1_001.fastq" "75_S75_L001_R2_001.fastq" [67] "76_S76_L001_R1_001.fastq" "76_S76_L001_R2_001.fastq" "77_S77_L001_R1_001.fastq" [70] "77_S77_L001_R2_001.fastq" "85_S85_L001_R1_001.fastq" "85_S85_L001_R2_001.fastq" [73] "86_S86_L001_R1_001.fastq" "86_S86_L001_R2_001.fastq" "87_S87_L001_R1_001.fastq" [76] "87_S87_L001_R2_001.fastq" "88_S88_L001_R1_001.fastq" "88_S88_L001_R2_001.fastq" [79] "89_S89_L001_R1_001.fastq" "89_S89_L001_R2_001.fastq"

filtFs <- file.path(path, "filtered", paste0(sample.names, "_F_filt.fastq.gz"))
filtRs <- file.path(path, "filtered", paste0(sample.names, "_R_filt.fastq.gz"))

# test
out <- filterAndTrim(fnFs, filtFs, fnRs, filtRs, 
              truncLen=c(280,200),
              maxN=0, 
              maxEE=c(2,2), 
              truncQ=2, 
              rm.phix=TRUE,
              compress=TRUE, 
              multithread=TRUE) # On Windows set multithread=FALSE

For every parameter, I've tested to turn it off (#) or increase e.g. maxN=999... etc.

Message remains: Warning message: In filterAndTrim(fnFs, filtFs, fnRs, filtRs, truncLen = c(280, 200), : No reads passed the filter. Please revisit your filtering parameters.

Here as you suggested:

out <- filterAndTrim(fnFs[[1]], filtFs[[1]], fnRs[[1]], filtRs[[1]],
              truncLen=c(280,200),
              maxN=0, 
              maxEE=c(2,2), 
              truncQ=2, 
              rm.phix=TRUE,
              compress=TRUE, 
              multithread=TRUE) 

The filter removed all reads: /s-schijf/fuentess/Data/Program/Run3_16S/Unzipped/filtered/1_F_filt.fastq.gz and /s-schijf/fuentess/Data/Program/Run3_16S/Unzipped/filtered/1_R_filt.fastq.gz not written. Warning messages: 1: In file.remove(fout[[1]]) : cannot remove file '/s-schijf/fuentess/Data/Program/Run3_16S/Unzipped/filtered/1_F_filt.fastq.gz', reason 'No such file or directory' 2: In file.remove(fout[[2]]) : cannot remove file '/s-schijf/fuentess/Data/Program/Run3_16S/Unzipped/filtered/1_R_filt.fastq.gz', reason 'No such file or directory' 3: In filterAndTrim(fnFs[[1]], filtFs[[1]], fnRs[[1]], filtRs[[1]], : No reads passed the filter. Please revisit your filtering parameters.

I'm happy to share my fastq files, in case there's something funky there... I must say, I get these from sequencing facilities, and in principle they should be fine.

Thanks!

[benjjneb]

Could you share the F/R fastq files from one sample that is failing in this way?

My email is benjamin DOT j DOT callahan AT gmail DOT com

[sfeds]

Done!

[benjjneb]

I'm not getting the same result with your example samples. See code below, if you run the same thing on the same sample, are you still getting the error?

path <- file.path("~/Desktop/sfed")
setwd(path)
fnF <- "1_S1_L001_R1_001.fastq.gz"
fnR <- "1_S1_L001_R2_001.fastq.gz"
filtF <- file.path("filtered", fnF)
filtR <- file.path("filtered", fnR)
library(dada2); packageVersion("dada2")

[1] ‘1.10.0’

filterAndTrim(fnF, filtF, fnR, filtR, truncLen=c(280, 200), maxN=0, maxEE=c(2,2), truncQ=2, verbose=TRUE)

Read in 143475 paired-sequences, output 105218 (73.3%) filtered paired-sequences.

[sfeds]

Hi, so I tried the all-time trick of uninstalling and reinstalling the package. I work in a server and they recently upgraded the R version, and I think I had some dependencies issues...

Anyway, happy to say, it all goes like a charm now! Thanks and sorry for not spotting this before!

[Brloajnek-a]

Hi, I am also having this issue, no reads are passing the filter, even though they are of good quality, and changing the parameters also doesn't help.

The first parameters I used: out <- filterAndTrim(fnFs, filtFs, fnRs, filtRs, truncLen=c(240,240), maxN=0, maxEE=c(2,2), truncQ=2, rm.phix=TRUE, compress=TRUE, multithread=FALSE)

The filter removed all reads: C:\Users\blank\Documents\Programming\R\Projects\Dada2\Raw\filtered\AD01_F_filt.fastq.gz and C:\Users\blank\Documents\Programming\R\Projects\Dada2\Raw\filtered\AD01_R_filt.fastq.gz not written.

I also cannot get the plot: plotQualityProfile(path) Error: BiocParallel errors 1 remote errors, element index: 1 0 unevaluated and other errors first remote error: 'x' contains missing values In addition: Warning message: In serialize(data, node$con) : 'package:stats' may not be available when loading

I've looked in Qiime2 at my reads, and they are good.

I guess I need to reinstall the package but I've only just installed it?

I would really appreciate any help. Thanks!

[benjjneb]

no reads are passing the filter, even though they are of good quality

Usually this is caused by truncLen set too high, higher than the length of the reads. What length are the reads in the fastq files you are applying filterAndTrim to?

I also cannot get the plot: plotQualityProfile(path) Error: BiocParallel errors

Honestly no idea what is going on there, but it looks like error in a supporting R library, not DADA2 code. Have you used this installation of R/packages in the past successfully?

[Brloajnek-a]

Hi,

thank you for the fast reply. All of my reads are 250 nt. I have uploaded to the original comment a better figure of my reads so it is visible that the quality is good and read length.

I have uninstalled and reinstalled the dada2, but the issue remains. I am adding all the warnings I get from the installation. warnings.txt

Thank you for your help.

[benjjneb]

The warnings in your text file are nothing to be concerned about. Just compiler warnings about comparisons between unsigned and signed integers mostly. The package looks like it built successfully.

What does the output of plotQualityProfile("C:\Users\blank\Documents\Programming\R\Projects\Dada2\Raw\filtered\AD01_F_filt.fastq.gz") look like?

[Brloajnek-a]

Hi Benjamin, it was actually a stupid mistake. I was looking at the length of raw reads with barcodes and primers. My demultiplexed reads are 200 nt and filtering is now working.

What does the output of plotQualityProfile("C:\Users\blank\Documents\Programming\R\Projects\Dada2\Raw\filtered\AD01_F_filt.fastq.gz") look like?

This again doesn't work, the same error as before.

But I will skip these plots and use the quality plots made in Qiime2 to decide on proper trimming and truncating. (I know I can run dada2 there also but it is on VirtualBox so moved to R to work in Windows to have more RAM available).

Thanks for your help and sorry for the confusion.

[WJH58]

Hi, so I tried the all-time trick of uninstalling and reinstalling the package. I work in a server and they recently upgraded the R version, and I think I had some dependencies issues...

Anyway, happy to say, it all goes like a charm now! Thanks and sorry for not spotting this before!

Hi, I had the same problem. How did you solve it finally? My R and dada2 are the latest version.