Parliament2 v0.1.8 VS v0.1.10 - performances

[MaestSi]

Hi Parliament2 developers, I am writing to ask for some explanations, since it looks like to me there has been quite a change in performances between versions 0.1.8 and 0.1.10 - towards higher precision and lower sensitivity - and I would like to know if this is something done on purpose and if anybody else experienced this too. I am running Parliament2 on a 30X bam file obtained mapping Illumina paired-end reads from NA24385 sample and then evaluating performances with Truvari based on a set of GIAB high-confidence calls (ftp://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/AshkenazimTrio/analysis/NIST_SVs_Integration_v0.6/). Here are my results: SV-type ||| Parliament2 v0.1.8 sens/prec ||| Parliament2 v0.1.10 sens/prec DELs ||| 69%/32% ||| 67%/64% DELs PASS ||| 69%/78% ||| 49%/83% INSs ||| 22%/53% ||| 22%/52% INSs PASS ||| 20%/91% ||| 20%/93%

In summary, I noticed that DELs (without PASS filtering) seem to be more precise, however when the PASS filtering is applied the sensitivity tends to drop from 69% to 49%. Is this more "conservative" filtering something wanted, or maybe is it a consequence of updating the versions of the callers without updating the weights accordingly? Moreover I was surprised by the fact that for Manta my results look very strong for DELs. I have to say that for Parliament2 I am not able to run breakseq2, since I am using hg38 reference genome, so my Parliament2 results might be negatively affected by this.

SV-type ||| Manta sens/prec DELs ||| 67%/92% DELs PASS ||| 63%/93% INSs ||| 22%/87% INSs PASS ||| 21%/88%

Thank you.

[AndrewCarroll]

Hi @MaestSi

Are the VCFs for your runs share-able. I could take a look by eye at the changes and see if anything looks like a regression from 0.1.8 to 0.1.10.

Manta is a strong individual caller, especially for smaller events. Within Parliament2, Manta provides the largest individual contribution to accuracy (though for larger events, it is surpassed by some of the other methods).

[MaestSi]

Sure, here they are! NA24385_30X.combined.genotyped_Parliament2_v0.1.8.txt NA24385_30X.combined.genotyped_Parliament2_v0.1.10.txt

[AndrewCarroll]

Hi @MaestSi

Thank you for reporting this. I am still looking into the issues and don't yet have an obvious answer. I noticed that the 1.10 calls seem to have fewer calls from MANTA, both as a whole and those which have PASS. I am looking into whether anything has changed with this that could cause calls to drop or be filtered. And I will report back again here.

Thanks, Andrew

[MaestSi]

Hi @AndrewCarroll , did you further inspect this issue, or do you have any updates about the Parliament2 project? Thanks, Simone