search

broadinstitute / picard

A set of command line tools (in Java) for manipulating high-throughput sequencing (HTS) data and formats such as SAM/BAM/CRAM and VCF.
https://broadinstitute.github.io/picard/
MIT License
984 stars 368 forks source link

CrosscheckFingerprints- No non-zero results found. This is likely an error #1577

Closed parvathisudha closed 4 years ago

parvathisudha commented 4 years ago

Hi,

I am trying to use Crosscheckfingerprints using Picard in HPC system. I am getting an error message as sample1 missing LEFT group sample 2 missing RIGHT group and Non-zero results found. I couldn't understand the error properly. Can you please explain how to solve this issue? Command used: java -jar /Application/picard/build/libs/picardcloud.jar CrosscheckFingerprints --INPUT /BAM/tumor1.bam --SECOND_INPUT /BAM/normal1.bam --HAPLOTYPE_MAP /Application/hg19_chr_map.txt --LOD_THRESHOLD -5 --OUTPUT /Crosscheck/sample1_crosscheck.out

Error: ERROR 2020-09-16 16:25:56 CrosscheckFingerprints sample 0661-179_0005063474_SALIVA_S3 is missing from LEFT group ERROR 2020-09-16 16:25:56 CrosscheckFingerprints sample 0661-179_4175575_CD138plus_S4 is missing from RIGHT group ERROR 2020-09-16 16:25:56 CrosscheckFingerprints No non-zero results found. This is likely an error. Probable cause: there are no reads or variants at fingerprinting sites

Log report: 16:22:25.428 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/geode2/home/../picard/build/libs/picardcloud.jar!/com/intel/gkl/native/libgkl_compression.so [Wed Sep 16 16:22:25 EDT 2020] CrosscheckFingerprints --INPUT /BAM/tumor1.bam --SECOND_INPUT /BAM/normal1.bam --OUTPUT /Crosscheck/sample1_crosscheck.out --HAPLOTYPE_MAP /Application/hg19_chr_map.txt --LOD_THRESHOLD -5.0 --CROSSCHECK_MODE CHECK_SAME_SAMPLE --CROSSCHECK_BY READGROUP --NUM_THREADS 1 --CALCULATE_TUMOR_AWARE_RESULTS true --ALLOW_DUPLICATE_READS false --GENOTYPING_ERROR_RATE 0.01 --OUTPUT_ERRORS_ONLY false --LOSS_OF_HET_RATE 0.5 --EXPECT_ALL_GROUPS_TO_MATCH false --EXIT_CODE_WHEN_MISMATCH 1 --EXIT_CODE_WHEN_NO_VALID_CHECKS 1 --MAX_EFFECT_OF_EACH_HAPLOTYPE_BLOCK 3.0 --TEST_INPUT_READABILITY true --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 5 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --GA4GH_CLIENT_SECRETS client_secrets.json --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false Sep 16, 2020 4:22:25 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine INFO: Failed to detect whether we are running on Google Compute Engine. [Wed Sep 16 16:22:25 EDT 2020] Executing as parkanha@c54 on Linux 3.10.0-1127.19.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_262-b10; Deflater: Intel; Inflater: Intel; Provider GCS is available; Picard version: Version:2.23.4-SNAPSHOT INFO 2020-09-16 16:22:25 CrosscheckFingerprints SECOND_INPUT is not empty, and CROSSCHECK_MODE==CHECK_SAME_SAMPLE. NOT doing cross-check. Will only compare each SAMPLE in INPUT against that sample in SECOND_INPUT. WARNING 2020-09-16 16:22:25 CrosscheckFingerprints CROSSCHECK_BY is not SAMPLE, This doesn't make sense in non-crosscheck mode. Setting CROSSCHECK_BY to SAMPLE. INFO 2020-09-16 16:22:26 CrosscheckFingerprints Fingerprinting 1 INPUT files. INFO 2020-09-16 16:22:26 FingerprintChecker Reading an indexed file (/BAM/tumor1.bam) INFO 2020-09-16 16:24:29 FingerprintChecker Processed files. 1 fingerprints found in map. INFO 2020-09-16 16:24:29 CrosscheckFingerprints Fingerprinting 1 SECOND_INPUT files. INFO 2020-09-16 16:24:29 FingerprintChecker Reading an indexed file (/BAM/normal1.bam) INFO 2020-09-16 16:25:55 FingerprintChecker Processed files. 1 fingerprints found in map. INFO 2020-09-16 16:25:55 CrosscheckFingerprints Checking each sample in INPUT with the same sample in SECOND_INPUT. ERROR 2020-09-16 16:25:56 CrosscheckFingerprints sample normal1 is missing from LEFT group ERROR 2020-09-16 16:25:56 CrosscheckFingerprints sample tumor1 is missing from RIGHT group ERROR 2020-09-16 16:25:56 CrosscheckFingerprints No non-zero results found. This is likely an error. Probable cause: there are no reads or variants at fingerprinting sites [Wed Sep 16 16:25:56 EDT 2020] picard.fingerprint.CrosscheckFingerprints done. Elapsed time: 3.51 minutes. Runtime.totalMemory()=2076049408 To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp (END)

Thanks Parvathi.

yfarjoun commented 4 years ago

you should use

--CHECK_MODE CROSSCHECK_ALL_OTHER since the name of your tumor (tumor1) is not the same as the name of your normal (normal1).

parvathisudha commented 4 years ago

I see. My samples are named as Sample1_xxx_tumor.bam and Sample1_xxx_normal.bam When I tried the option you mentioned, this is the error I get:

Error: Argument CROSSCHECK_MODE has a bad value: CROSSCHECK_ALL_OTHER. 'CROSSCHECK_ALL_OTHER' is not a valid value for CrosscheckMode. Allowed values are CHECK_SAME_SAMPLE (In this mode, each sample in INPUT will only be checked against a single corresponding sample in SECOND_INPUT. If a corresponding sample cannot be found, the program will proceed, but report the missing samples and return the value specified in EXIT_CODE_WHEN_MISMATCH. The corresponding samples are those that equal each other, after possible renaming via INPUT_SAMPLE_MAP and SECOND_INPUT_SAMPLE_MAP. In this mode CROSSCHECK_BY must be SAMPLE.) CHECK_ALL_OTHERS (In this mode, each sample in INPUT will be checked against all the samples in SECOND_INPUT.)

Is it possible to give the directory with all the bam files instead of single bam files ? Or For the INPUT and SECOND_INPUT arguments, can I give all the files together? Like, tumors sample names with file location as INPUT and Normal sample names with file location as SECOND_INPUT. Is that possible?

Thanks for the help!

Parvathi.

yfarjoun commented 4 years ago

No. But you can provide the input arguments multiple times and you can provide a list of File names as the input argument as well.

On Wed, Sep 16, 2020 at 7:42 PM parvathisudha notifications@github.com wrote:

I see. Is it possible to give the directory with all the bam files instead of single bam files ?

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/broadinstitute/picard/issues/1577#issuecomment-693723251, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU6JUTQQN4KM4WX5AX6YBDSGFEOBANCNFSM4RPMJ2NA .

parvathisudha commented 4 years ago

Hi,

Crosscheckfingerprints run without errors. But, I am comparing the Tumor and Normal samples from same individual (as per the details I got from our sequencing core). While using NGSCheckMate, they were identified as matched pairs. But Crosscheckfingerprints says they don't match. "CrosscheckFingerprints 1 READGROUPs did not relate as expected". I am getting the similar output for all the samples.

Is there any other parameter I should consider while running this?

Commad used: java -jar /Application/picard/build/libs/picardcloud.jar CrosscheckFingerprints --INPUT /BAM/0661-181_CD138plus_S4_final.bam --SECOND_INPUT /BAM/0661-181_SALIVA_S3_final.bam --HAPLOTYPE_MAP Application/hg19_chr_map_v2.txt --CROSSCHECK_MODE CHECK_ALL_OTHERS --OUTPUT Crosscheck/0661_181_crosscheck.out

Metrics output

METRICS CLASS picard.fingerprint.CrosscheckMetric

LEFT_GROUP_VALUE RIGHT_GROUP_VALUE RESULT DATA_TYPE LOD_SCORE LOD_SCORE_TUMOR_NORMAL LOD_SCORE_NORMAL_TUMOR LEFT_RUN_BARCODE LEFT_LANE LEFT_MOLECULAR_BARCODE_SEQUENCE LEFT_LIBRARY LEFT_SAMPLE LEFT_FILE RIGHT_RUN_BARCODE RIGHT_LANE RIGHT_MOLECULAR_BARCODE_SEQUENCE RIGHT_LIBRARY RIGHT_SAMPLE RIGHT_FILE DNAseq49 DNAseq49 UNEXPECTED_MATCH READGROUP 461.176766 465.302021 464.499576 ? -1 ? myeloma_panel 0661-181_CD138plus_S4 file:///BAM/0661-181_CD138plus_S4_final.bam ? -1 ? myeloma_panel 0661-181_SALIVA_S3 file:///BAM/0661-181_SALIVA_S3_final.bam

Log report:

CrosscheckFingerprints --INPUT /BAM/0661-181_CD138plus_S4_final.bam --SECOND_INPUT BAM/0661-181_SALIVA_S3_final.bam --CROSSCHECK_MODE CHECK_ALL_OTHERS --OUTPUT Crosscheck/0661_181_crosscheck.out --HAPLOTYPE_MAP Application/hg19_chr_map_v2.txt --LOD_THRESHOLD 0.0 --CROSSCHECK_BY READGROUP --NUM_THREADS 1 --CALCULATE_TUMOR_AWARE_RESULTS true --ALLOW_DUPLICATE_READS false --GENOTYPING_ERROR_RATE 0.01 --OUTPUT_ERRORS_ONLY false --LOSS_OF_HET_RATE 0.5 --EXPECT_ALL_GROUPS_TO_MATCH false --EXIT_CODE_WHEN_MISMATCH 1 --EXIT_CODE_WHEN_NO_VALID_CHECKS 1 --MAX_EFFECT_OF_EACH_HAPLOTYPE_BLOCK 3.0 --TEST_INPUT_READABILITY true --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 5 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --GA4GH_CLIENT_SECRETS client_secrets.json --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false Sep 17, 2020 10:25:10 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine INFO: Failed to detect whether we are running on Google Compute Engine. [Thu Sep 17 10:25:10 EDT 2020] Executing as parkanha@c9 on Linux 3.10.0-1127.19.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_262-b10; Deflater: Intel; Inflater: Intel; Provider GCS is available; Picard version: Version:2.23.4-SNAPSHOT INFO 2020-09-17 10:25:10 CrosscheckFingerprints SECOND_INPUT is not empty, and CROSSCHECK_MODE==CHECK_ALL_OTHERS. Will compare fingerprints from INPUT against all the fingerprints in SECOND_INPUT. INFO 2020-09-17 10:25:11 CrosscheckFingerprints Fingerprinting 1 INPUT files. WARNING: BAM index file /BAM/0661-181_CD138plus_S4_final.bai is older than /BAM BAM/0661-181_CD138plus_S4_final.bam INFO 2020-09-17 10:25:11 FingerprintChecker Reading an indexed file (BAM/0661-181_CD138plus_S4_final.bam) INFO 2020-09-17 10:27:22 FingerprintChecker Processed files. 1 fingerprints found in map. INFO 2020-09-17 10:27:23 CrosscheckFingerprints Fingerprinting 1 SECOND_INPUT files. WARNING: BAM index file /BAM/0661-181_SALIVA_S3_final.bai is older than BAM /BAM/0661-181 _SALIVA_S3_final.bam INFO 2020-09-17 10:27:23 FingerprintChecker Reading an indexed file (BAM/0661-181 _SALIVA_S3_final.bam) INFO 2020-09-17 10:29:11 FingerprintChecker Processed files. 1 fingerprints found in map. INFO 2020-09-17 10:29:11 CrosscheckFingerprints Checking each READGROUP in INPUT with each READGROUP in SECOND_INPUT. WARNING 2020-09-17 10:29:12 CrosscheckFingerprints 1 READGROUPs did not relate as expected. [Thu Sep 17 10:29:12 EDT 2020] picard.fingerprint.CrosscheckFingerprints done. Elapsed time: 4.03 minutes. Runtime.totalMemory()=2076049408 To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp

yfarjoun commented 4 years ago

You are getting an "UNEXPECTED_MATCH" which was "unexpected" since the sample names were different, but is still indicating that you have a match....if you want the code to expect that all samples should match you can use --EXPECT_ALL_GROUPS_TO_MATCH true, or alternatively create a sample_map file and provide it via the SECOND_INPUT_SAMPLE_MAP argument (for example)

Please read the documentation for those arguments.

You are making me very happy that someone is trying out this tool!

On Thu, Sep 17, 2020 at 1:51 PM parvathisudha notifications@github.com wrote:

Hi,

Crosscheckfingerprints run without errors. But, I am comparing the Tumor and Normal samples from same individual (as per the details I got from our sequencing core). While using NGSCheckMate, they were identified as matched pairs. But Crosscheckfingerprints says they don't match. "CrosscheckFingerprints 1 READGROUPs did not relate as expected". I am getting the similar output for all the samples.

Is there any other parameter I should consider while running this?

Commad used: java -jar /Application/picard/build/libs/picardcloud.jar CrosscheckFingerprints --INPUT /BAM/0661-181_CD138plus_S4_final.bam --SECOND_INPUT /BAM/0661-181_SALIVA_S3_final.bam --HAPLOTYPE_MAP Application/hg19_chr_map_v2.txt --CROSSCHECK_MODE CHECK_ALL_OTHERS --OUTPUT Crosscheck/0661_181_crosscheck.out

Metrics output METRICS CLASS picard.fingerprint.CrosscheckMetric

LEFT_GROUP_VALUE RIGHT_GROUP_VALUE RESULT DATA_TYPE LOD_SCORE LOD_SCORE_TUMOR_NORMAL LOD_SCORE_NORMAL_TUMOR LEFT_RUN_BARCODE LEFT_LANE LEFT_MOLECULAR_BARCODE_SEQUENCE LEFT_LIBRARY LEFT_SAMPLE LEFT_FILE RIGHT_RUN_BARCODE RIGHT_LANE RIGHT_MOLECULAR_BARCODE_SEQUENCE RIGHT_LIBRARY RIGHT_SAMPLE RIGHT_FILE DNAseq49 DNAseq49 UNEXPECTED_MATCH READGROUP 461.176766 465.302021 464.499576 ? -1 ? myeloma_panel 0661-181_CD138plus_S4 file:///BAM/0661-181_CD138plus_S4_final.bam ? -1 ? myeloma_panel 0661-181_SALIVA_S3 file:///BAM/0661-181_SALIVA_S3_final.bam

Log report:

CrosscheckFingerprints --INPUT /BAM/0661-181_CD138plus_S4_final.bam --SECOND_INPUT BAM/0661-181_SALIVA_S3_final.bam --CROSSCHECK_MODE CHECK_ALL_OTHERS --OUTPUT Crosscheck/0661_181_crosscheck.out --HAPLOTYPE_MAP Application/hg19_chr_map_v2.txt --LOD_THRESHOLD 0.0 --CROSSCHECK_BY READGROUP --NUM_THREADS 1 --CALCULATE_TUMOR_AWARE_RESULTS true --ALLOW_DUPLICATE_READS false --GENOTYPING_ERROR_RATE 0.01 --OUTPUT_ERRORS_ONLY false --LOSS_OF_HET_RATE 0.5 --EXPECT_ALL_GROUPS_TO_MATCH false --EXIT_CODE_WHEN_MISMATCH 1 --EXIT_CODE_WHEN_NO_VALID_CHECKS 1 --MAX_EFFECT_OF_EACH_HAPLOTYPE_BLOCK 3.0 --TEST_INPUT_READABILITY true --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 5 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --GA4GH_CLIENT_SECRETS client_secrets.json --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false Sep 17, 2020 10:25:10 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine INFO: Failed to detect whether we are running on Google Compute Engine. [Thu Sep 17 10:25:10 EDT 2020] Executing as parkanha@c9 on Linux 3.10.0-1127.19.1.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_262-b10; Deflater: Intel; Inflater: Intel; Provider GCS is available; Picard version: Version:2.23.4-SNAPSHOT INFO 2020-09-17 10:25:10 CrosscheckFingerprints SECOND_INPUT is not empty, and CROSSCHECK_MODE==CHECK_ALL_OTHERS. Will compare fingerprints from INPUT against all the fingerprints in SECOND_INPUT. INFO 2020-09-17 10:25:11 CrosscheckFingerprints Fingerprinting 1 INPUT files. WARNING: BAM index file /BAM/0661-181_CD138plus_S4_final.bai is older than /BAM BAM/0661-181_CD138plus_S4_final.bam INFO 2020-09-17 10:25:11 FingerprintChecker Reading an indexed file (BAM/0661-181_CD138plus_S4_final.bam) INFO 2020-09-17 10:27:22 FingerprintChecker Processed files. 1 fingerprints found in map. INFO 2020-09-17 10:27:23 CrosscheckFingerprints Fingerprinting 1 SECOND_INPUT files. WARNING: BAM index file /BAM/0661-181_SALIVA_S3_final.bai is older than BAM /BAM/0661-181 _SALIVA_S3_final.bam INFO 2020-09-17 10:27:23 FingerprintChecker Reading an indexed file (BAM/0661-181 _SALIVA_S3_final.bam) INFO 2020-09-17 10:29:11 FingerprintChecker Processed files. 1 fingerprints found in map. INFO 2020-09-17 10:29:11 CrosscheckFingerprints Checking each READGROUP in INPUT with each READGROUP in SECOND_INPUT. WARNING 2020-09-17 10:29:12 CrosscheckFingerprints 1 READGROUPs did not relate as expected. [Thu Sep 17 10:29:12 EDT 2020] picard.fingerprint.CrosscheckFingerprints done. Elapsed time: 4.03 minutes. Runtime.totalMemory()=2076049408 To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/broadinstitute/picard/issues/1577#issuecomment-694397439, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU6JUTSBJVB25OFMLXVPELSGJEBDANCNFSM4RPMJ2NA .

parvathisudha commented 4 years ago

Thanks for the help! It's working fine.

parvathisudha commented 4 years ago

Hi, I have another question. Is there a way to output the metrics for each sample along with the metrics statistics?

yfarjoun commented 4 years ago

not sure what you mean...but could you close this ticket if it's resolved, and open a new issue with your new question?

On Mon, Sep 21, 2020 at 9:44 AM parvathisudha notifications@github.com wrote:

Hi, I have another question. Is there a way to output the metrics for each sample along with the metrics statistics?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/broadinstitute/picard/issues/1577#issuecomment-696124279, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAU6JUUN2DIJVKQGEKXNWDDSG5KFPANCNFSM4RPMJ2NA .