search

bcbio / bcbio-nextgen

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis
https://bcbio-nextgen.readthedocs.io
MIT License
990 stars 354 forks source link

errors in bam #1215

Closed parlar closed 8 years ago

parlar commented 8 years ago

Running $ picard ValidateSamFile INPUT=xxxxx-ready.bam (bam file produced by bcbio-nextgen) gives the error listed below. Is this due to bam files being merged? Any potential problems from this, reason to worry?

Adam Ewing, who makes BamSurgeon pointed this out.

[Fri Feb 05 10:35:56 CET 2016] picard.sam.ValidateSamFile INPUT=./calling/final/15-8571/15-8571-ready.bam    MODE=VERBOSE MAX_OUTPUT=100 IGNORE_WARNINGS=false VALIDATE_INDEX=true IS_BISULFITE_SEQUENCED=false MAX_OPEN_TEMP_FILES=8000 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
[Fri Feb 05 10:35:56 CET 2016] Executing as genetik@v01s979.vll.se on Linux 3.10.0-327.4.4.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_65-b17; Picard version: 1.141(8ece590411350163e7689e9e77aab8efcb622170_1447695087) IntelDeflater
ERROR: Record 7979, Read name M00568:145:AM0EH:1:1118:28619:18261, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 7980, Read name M00568:145:AM0EH:1:1118:28619:18261, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 7980, Read name M00568:145:AM0EH:1:1118:28619:18261, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 7979, Read name M00568:145:AM0EH:1:1118:28619:18261, Both mates are marked as first of pair
ERROR: Record 74063, Read name M00568:145:AM0EH:1:1105:25763:21317, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 74065, Read name M00568:145:AM0EH:1:1105:25763:21317, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 74065, Read name M00568:145:AM0EH:1:1105:25763:21317, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 74063, Read name M00568:145:AM0EH:1:1105:25763:21317, Both mates are marked as first of pair
ERROR: Record 97668, Read name M00568:145:AM0EH:1:1107:11206:14110, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 97724, Read name M00568:145:AM0EH:1:1107:11206:14110, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 97724, Read name M00568:145:AM0EH:1:1107:11206:14110, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 97668, Read name M00568:145:AM0EH:1:1107:11206:14110, Both mates are marked as first of pair
ERROR: Record 117910, Read name M00568:145:AM0EH:1:1109:28056:8208, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 117911, Read name M00568:145:AM0EH:1:1109:28056:8208, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 117911, Read name M00568:145:AM0EH:1:1109:28056:8208, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 117910, Read name M00568:145:AM0EH:1:1109:28056:8208, Both mates are marked as first of pair
ERROR: Record 118098, Read name M00568:145:AM0EH:1:1109:11232:6691, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 118099, Read name M00568:145:AM0EH:1:1109:11232:6691, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 118099, Read name M00568:145:AM0EH:1:1109:11232:6691, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 118098, Read name M00568:145:AM0EH:1:1109:11232:6691, Both mates are marked as first of pair
ERROR: Record 138749, Read name M00568:145:AM0EH:1:1113:10503:17115, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 138750, Read name M00568:145:AM0EH:1:1113:10503:17115, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 138750, Read name M00568:145:AM0EH:1:1113:10503:17115, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 138749, Read name M00568:145:AM0EH:1:1113:10503:17115, Both mates are marked as first of pair
ERROR: Record 223524, Read name M00568:145:AM0EH:1:2103:26642:5302, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 223662, Read name M00568:145:AM0EH:1:2103:26642:5302, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 223662, Read name M00568:145:AM0EH:1:2103:26642:5302, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 223524, Read name M00568:145:AM0EH:1:2103:26642:5302, Both mates are marked as first of pair
ERROR: Record 323929, Read name M00568:145:AM0EH:1:2102:24925:19143, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 323930, Read name M00568:145:AM0EH:1:2102:24925:19143, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 323930, Read name M00568:145:AM0EH:1:2102:24925:19143, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 323929, Read name M00568:145:AM0EH:1:2102:24925:19143, Both mates are marked as first of pair
ERROR: Record 444630, Read name M00568:145:AM0EH:1:1110:27417:20278, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 444823, Read name M00568:145:AM0EH:1:1110:27417:20278, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 444823, Read name M00568:145:AM0EH:1:1110:27417:20278, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 444630, Read name M00568:145:AM0EH:1:1110:27417:20278, Both mates are marked as first of pair
ERROR: Record 569855, Read name M00568:145:AM0EH:1:2101:8957:10090, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 569856, Read name M00568:145:AM0EH:1:2101:8957:10090, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 569856, Read name M00568:145:AM0EH:1:2101:8957:10090, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 569855, Read name M00568:145:AM0EH:1:2101:8957:10090, Both mates are marked as first of pair
ERROR: Record 652702, Read name M00568:145:AM0EH:1:2110:24707:14957, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 652704, Read name M00568:145:AM0EH:1:2110:24707:14957, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 652704, Read name M00568:145:AM0EH:1:2110:24707:14957, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 652702, Read name M00568:145:AM0EH:1:2110:24707:14957, Both mates are marked as first of pair
ERROR: Record 656602, Read name M00568:145:AM0EH:1:2113:29297:16539, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 656603, Read name M00568:145:AM0EH:1:2113:29297:16539, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 656603, Read name M00568:145:AM0EH:1:2113:29297:16539, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 656602, Read name M00568:145:AM0EH:1:2113:29297:16539, Both mates are marked as first of pair
ERROR: Record 805041, Read name M00568:145:AM0EH:1:1119:15851:18460, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 805042, Read name M00568:145:AM0EH:1:1119:15851:18460, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 805042, Read name M00568:145:AM0EH:1:1119:15851:18460, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 805041, Read name M00568:145:AM0EH:1:1119:15851:18460, Both mates are marked as first of pair
ERROR: Record 819781, Read name M00568:145:AM0EH:1:1113:18409:4146, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819784, Read name M00568:145:AM0EH:1:1113:18409:4146, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819784, Read name M00568:145:AM0EH:1:1113:18409:4146, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819781, Read name M00568:145:AM0EH:1:1113:18409:4146, Both mates are marked as first of pair
ERROR: Record 819796, Read name M00568:145:AM0EH:1:2110:7399:17403, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819797, Read name M00568:145:AM0EH:1:2110:7399:17403, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819797, Read name M00568:145:AM0EH:1:2110:7399:17403, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819796, Read name M00568:145:AM0EH:1:2110:7399:17403, Both mates are marked as first of pair
ERROR: Record 819881, Read name M00568:145:AM0EH:1:1103:3753:6208, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819882, Read name M00568:145:AM0EH:1:1103:3753:6208, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819882, Read name M00568:145:AM0EH:1:1103:3753:6208, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819881, Read name M00568:145:AM0EH:1:1103:3753:6208, Both mates are marked as first of pair
ERROR: Record 819887, Read name M00568:145:AM0EH:1:2117:20900:22386, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819893, Read name M00568:145:AM0EH:1:2117:20900:22386, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819893, Read name M00568:145:AM0EH:1:2117:20900:22386, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819887, Read name M00568:145:AM0EH:1:2117:20900:22386, Both mates are marked as first of pair
ERROR: Record 819905, Read name M00568:145:AM0EH:1:2102:8672:13686, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819935, Read name M00568:145:AM0EH:1:2102:8672:13686, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819935, Read name M00568:145:AM0EH:1:2102:8672:13686, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819905, Read name M00568:145:AM0EH:1:2102:8672:13686, Both mates are marked as first of pair
ERROR: Record 819907, Read name M00568:145:AM0EH:1:2102:8672:13807, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819937, Read name M00568:145:AM0EH:1:2102:8672:13807, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819937, Read name M00568:145:AM0EH:1:2102:8672:13807, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819907, Read name M00568:145:AM0EH:1:2102:8672:13807, Both mates are marked as first of pair
ERROR: Record 819917, Read name M00568:145:AM0EH:1:2113:21600:14709, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819970, Read name M00568:145:AM0EH:1:2113:21600:14709, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819970, Read name M00568:145:AM0EH:1:2113:21600:14709, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819917, Read name M00568:145:AM0EH:1:2113:21600:14709, Both mates are marked as first of pair
ERROR: Record 819919, Read name M00568:145:AM0EH:1:2113:21746:15141, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819981, Read name M00568:145:AM0EH:1:2113:21746:15141, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819981, Read name M00568:145:AM0EH:1:2113:21746:15141, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819919, Read name M00568:145:AM0EH:1:2113:21746:15141, Both mates are marked as first of pair
ERROR: Record 819921, Read name M00568:145:AM0EH:1:2113:21854:15193, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819999, Read name M00568:145:AM0EH:1:2113:21854:15193, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 819999, Read name M00568:145:AM0EH:1:2113:21854:15193, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819921, Read name M00568:145:AM0EH:1:2113:21854:15193, Both mates are marked as first of pair
ERROR: Record 819915, Read name M00568:145:AM0EH:1:2113:21860:14704, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 820004, Read name M00568:145:AM0EH:1:2113:21860:14704, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 820004, Read name M00568:145:AM0EH:1:2113:21860:14704, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819915, Read name M00568:145:AM0EH:1:2113:21860:14704, Both mates are marked as first of pair
ERROR: Record 819909, Read name M00568:145:AM0EH:1:2113:21988:14500, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 820033, Read name M00568:145:AM0EH:1:2113:21988:14500, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 820033, Read name M00568:145:AM0EH:1:2113:21988:14500, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819909, Read name M00568:145:AM0EH:1:2113:21988:14500, Both mates are marked as first of pair
ERROR: Record 819913, Read name M00568:145:AM0EH:1:2113:21992:14684, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 820035, Read name M00568:145:AM0EH:1:2113:21992:14684, Mate unmapped flag does not match read unmapped flag of mate
ERROR: Record 820035, Read name M00568:145:AM0EH:1:2113:21992:14684, Mate CIGAR string does not match CIGAR string of mate
ERROR: Record 819913, Read name M00568:145:AM0EH:1:2113:21992:14684, Both mates are marked as first of pair
Maximum output of [100] errors reached.
[Fri Feb 05 10:36:02 CET 2016] picard.sam.ValidateSamFile done. Elapsed time: 0.10 minutes.
Runtime.totalMemory()=669515776
To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
chapmanb commented 8 years ago

Pär; Thanks for checking in on BAM correctness. Are you noticing problems processing the BAM downstream? I've always found ValidateSamFile to be too stringent so don't work with it regularly. With test datasets I can replicate getting CIGAR string reports when using bwa on pretty vanilla inputs but not issues with both mates being marked as first.

If it's worth digging into more, I'd likely need to know more about your setup to try and replicate: what type of inputs do you have (standard fastq files, previously aligned BAMs, how long are the reads), what does your configuration look like (are you doing split alignments, recalibration/realignment?), what genome are you aligning against (hg19 versus hg38). Happy to try and help more if this is causing problems.

parlar commented 8 years ago

As far as I know there are no problems, I just though I should inform you if the results would be a reason for concern. But of course you already knew about this..

chapmanb commented 8 years ago

Pär; Thanks much. If you run into anything or identify issues happy to try and debug more. Closing for now but re-open if anything comes up.