search

stat-lab / EvalSVcallers

Evaluate the performances (precision and recall) of structural variation (SV) callers
32 stars 13 forks source link

Evaluation of Delly SV calls vs reference set NA12878 shows all zeros for precision and recall in the output (evaluate_SV_callers.pl) #8

Closed bioinfo89 closed 4 years ago

bioinfo89 commented 4 years ago

Hi, I am using evaluate_SV_callers.pl script to evaluate SV calls from Delly with Na12878. When I run the script, I get 0 for precision and recall for all the size ranges, I am not sure what went wrong here:

DEL

            <Number of supporting reads>
            2       3       4       5       6       7       8       9       10      12

Call (A) 247 175 145 132 120 110 104 99 96 87
Recall (A) 0 0 0 0 0 0 0 0 0 0
Precis (A) 0 0 0 0 0 0 0 0 0 0
Call (SS) 0 0 0 0 0 0 0 0 0 0
Recall (SS) 0 0 0 0 0 0 0 0 0 0
Precis (SS) 0 0 0 0 0 0 0 0 0 0
Call (S) 63 52 47 43 41 37 36 36 35 32
Recall (S) 0 0 0 0 0 0 0 0 0 0
Precis (S) 0 0 0 0 0 0 0 0 0 0
Call (M) 166 115 92 86 77 71 66 61 59 54
Recall (M) 0 0 0 0 0 0 0 0 0 0
Precis (M) 0 0 0 0 0 0 0 0 0 0
Call (L) 18 8 6 3 2 2 2 2 2 1
Recall (L) 0 0 0 0 0 0 0 0 0 0
Precis (L) 0 0 0 0 0 0 0 0 0 0

DUP

            <Number of supporting reads>
            2       3       4       5       6       7       8       9       10      12

Call (A) 171 131 110 95 85 72 63 58 56 51
Recall (A) 0 0 0 0 0 0 0 0 0 0
Precis (A) 0 0 0 0 0 0 0 0 0 0
Call (S) 17 14 11 9 7 6 5 5 5 4
Recall (S) 0 0 0 0 0 0 0 0 0 0
Precis (S) 0 0 0 0 0 0 0 0 0 0
Call (M) 136 105 88 76 69 59 51 47 47 45
Recall (M) 0 0 0 0 0 0 0 0 0 0
Precis (M) 0 0 0 0 0 0 0 0 0 0
Call (L) 18 12 11 10 9 7 7 6 4 2
Recall (L) 0 0 0 0 0 0 0 0 0 0
Precis (L) 0 0 0 0 0 0 0 0 0 0

Any help would be appreciated. Thanks!

stat-lab commented 4 years ago

The problem was attributed to the reference chromosome names based on the hg19 reference, which has 'chr' prefix in the chromosome names. This evaluation system has been developed with the GRCh37 reference. I changed all the scripts so that the hg19-based files can be used for the evaluation.

bioinfo89 commented 4 years ago

Dear Kosugi,

Though you have closed the issue, I am still getting all 0s while using evaluate_SV_callers.pl for Delly, Manta and CNVnator converted VCFs, since I have use hg19 as reference sequence, for lumpy it works fine. Not sure what went wrong.

Can you please look into this? Thanks!

Best, Khyati Chandratre

On 4 Aug 2020, at 8:49 PM, stat-lab notifications@github.com wrote:

Closed #8 https://github.com/stat-lab/EvalSVcallers/issues/8.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stat-lab/EvalSVcallers/issues/8#event-3622653541, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIAOPF6G2FNAASOYAAXA7QTR7C3DXANCNFSM4PSZOTCA.

stat-lab commented 4 years ago

Dear Khyati,

The script ‘evaluate_SV_callers.pl’ was updated to deal with the hg19-based vcf files. Please try it.

Shunichi

2020/08/08 5:59、bioinfo89 <notifications@github.com mailto:notifications@github.com>のメール:

Dear Kosugi,

Though you have closed the issue, I am still getting all 0s while using evaluate_SV_callers.pl for Delly, Manta and CNVnator converted VCFs, since I have use hg19 as reference sequence, for lumpy it works fine. Not sure what went wrong.

Can you please look into this? Thanks!

Best, Khyati Chandratre

On 4 Aug 2020, at 8:49 PM, stat-lab <notifications@github.com mailto:notifications@github.com> wrote:

Closed #8 <https://github.com/stat-lab/EvalSVcallers/issues/8 https://github.com/stat-lab/EvalSVcallers/issues/8>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/stat-lab/EvalSVcallers/issues/8#event-3622653541 https://github.com/stat-lab/EvalSVcallers/issues/8#event-3622653541>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIAOPF6G2FNAASOYAAXA7QTR7C3DXANCNFSM4PSZOTCA https://github.com/notifications/unsubscribe-auth/AIAOPF6G2FNAASOYAAXA7QTR7C3DXANCNFSM4PSZOTCA>.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/stat-lab/EvalSVcallers/issues/8#issuecomment-670710712, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADIBV3DLYPXWVFT7DNUSAGDR7RTL3ANCNFSM4PSZOTCA.