eblerjana
commented
2 years ago Hi Joe,
the pre-filtering is done in order to remove positions for which too the input haplotypes carry too many missing alleles. Since this haplotype information is important for PanGenie to make accurate positions, we typically remove sites for which more than 20% of the haplotypes carry a missing allele (note that there is a typo in the HPRC preprint, the 80% should be 20%, as we used the same strategy used for the HGSVC project). The filtering based on vcfbub, as well as the decomposition approach are specific to the HPRC graphs, more specifically to the VCFs generated based on snarl traversals of these graphs. The reason why the decomposition is needed for downstream analyses is described in detail in the preprint, and is done after genotyping bubbles with PanGenie. The approach used to filter the final genotypes helps to remove poor quality genotypes computed for a cohort of samples. It uses metrics like the mendelian consistency, genotype quality or the re-genotyping performance of samples present in the graph. Therefore, this approach only works if genotypes of a large cohort (like the 1000 Genomes samples) are available, that include trios. The pipelines used for HPRC genotyping and filtering are available at: https://bitbucket.org/jana_ebler/hprc-experiments/src/master/genotyping-experiments/, including the regression-based filtering pipeline for genotypes of the 1000 Genomes samples (https://bitbucket.org/jana_ebler/hprc-experiments/src/master/genotyping-experiments/workflow/rules/population-evaluation.smk). The latter computes the statistics needed for the regression, and runs the final filtering script that applies this model (https://bitbucket.org/jana_ebler/hprc-experiments/src/master/genotyping-experiments/workflow/scripts/analysis.py). Parameters used in this script might have to be adapted if applied to a cohorts of different sizes.
If you have genotyping data of a larger cohort (> 100 samples), I'd suggest using a similar regression-based filtering approach as we used for HPRC. If you want to filter genotypes computed for single samples, I'd suggest to use the genotype quality value for filtering. We typically use a threshold of 200 to extract relatively strict sets of high quality genotypes, but smaller values can be used to generate more lenient sets.
I hope this helps.
Best, Jana
JosephLalli
JD12138
Wangchangsh
Hi there,
Thanks for this great tool. I noticed that, in both the HPRC reference release paper and the original paper describing this tool, you employ complex pre and post pangenie variant filtering:
"We used vcfbub (version 0.1.0, (Garrison, 2021)) with parameters -l 0 and -r 100000 in order to filter the VCF..." "We additionally remove all records for which more than 80% of all 88 haplotypes carry a missing allele..." "we have implemented a decomposition approach which aims at detecting all variant alleles nested inside of multi-allelic top-level records..." "[We] defined a high quality subset of SV alleles that we can reliably genotype. For this purpose, we applied a machine learning approach similar to what we have presented previously..."
How important are these methods to the quality/reliability of your final genotypes, especially for large SV (> 50bp)? Does this package implement these filters, and if not, is there code that does? Do you have pre-trained models to use for the machine learning/regression based filter?
Just trying to get high-quality SV genotypes on my diverse short-read dataset, so any help is greatly appreciated.
Thanks again! -Joe