broadinstitute / gatk

Official code repository for GATK versions 4 and up
https://software.broadinstitute.org/gatk
Other
1.65k stars 581 forks source link

Non intuitive behaveour of VariantFiltration ```--invalidate-previous-filters``` #7608

Open vruano opened 2 years ago

vruano commented 2 years ago

Affected tool(s) or class(es)

VariantFiltration

Affected version(s)

Master 2021-01 onwards.

Description

VariantFiltration's --invalidate-previous-filters prevent any output variant to be marked as PASS even if it passes the filters that are passed to that run of the filtration tool. What I would expect is that variants that pass the current set of filters will be marked as PASS regarless whether they were filtered or not filtered (PASS, or '.') in the input.

Perhaps the case is that this option is not meant.to be used with current/new filters being specified as to provide a way to revert previous filtration, but in that case it should fail if new filters are specified.

Steps to reproduce

Just try it out with a VCF with some filters already applied and run VF with additional filters that will not result in all variants to be filtered.

You will see that no output variant is set To PASS but rather are kept as unknown '.'

Expected behavior

Either it does fail if a new filter is applied in the same tool run or passing variants filter are setlp to Pw.

Actual behavior

Described above, no error message and no pass variants only "." ones.

vruano commented 2 years ago

Looking at the code changes the way this argument work has changed from simply clearing previous filters while apply new ones to being used standalone just to revert filtering (i.e. adding the . again) with the expectation that it would not be used in conjuction with new filters.

I think we should create to different arguments for each of those operations. say --overwrite-existing-filters that that the former and --revert-filters that that the latter and will fail if you try to filter again in the same run. ... or perhaps the latter should have its own tool RevertFilration.

lbergelson commented 2 years ago

@vruano Do you have bandwidth to fix this?