ONP rendering breaks the OxoG filter (Part I -- re-annotation of TCGA MAF and annotation overwriting)

[LeeTL1220]

The filter cannot handle values with "|". These are generated in the columns that the OxoG filter needs in the case of ONPs (e.g. t_alt_count), since there are different values for each base. Oncotator behavior is correct, but we need more functionality to make sure that an option exists to run a TCGA MAF through the OxoG filter -- with minimal changes to the oxoG filter code.

There are several possible solutions for this issue:

• allow user to specify columns to collapse (via averaging or first value etc).
• implement easy re-annotation of TCGA MAF files in Oncotator.
• create a separate postprocessing tool that handles the columns in question.
• Run the TCGA MAF twice. One annotation run without --infer-onps, then oxoG filter, then cut columns (as described in GATK oncotator forum), and then re-annotate.

If one of the latter two are chosen, this is not really an oncotator issue, but we can track it here. Note that only the third choice handles all edge cases (e.g. we have a DNP where one of the two mutations is an artifact)

[LeeTL1220]

Note that easy re-annotation of TCGA MAF (second bullet point), could just be to cut the columns inside oncotator.

[LeeTL1220]

FYI... @kcibul and @ldgauthier

I have already discussed with @kcibul and he seems to be on board.

After discussion with Chip @chipstewart the solution to support M2 in FH is as follows:

1. M2 produces VCF
2. Oncotator converts and annotates VCF to TCGA MAF without --infer-onps (uses full datasource set per @kcibul request)
3. OxoG filter runs
4. Re-annotate with infer-onps and NEW functionality that collapses xNP fields.

How to collapse fields for xNPs: Assume that a field comes in as "x|y|...". For counts:

• final number is the min of the counts.
• Run a fisher exact test for the read counts. If significant (p<0.05), display warning.
• Make sure to preserve original value in [count_field_name]_full

For other fields: just take the mean

[ldgauthier]

Sounds good.

[LeeTL1220]

For my own reference (unit tests still to be created) only for the TCGA MAF to TCGA MAF component. Backing code mostly built as of this writing (not shown here) and all regression tests pass, so far.

Still needed to be built:

• [x] Prioritization against "INPUT" annotations when --prune-tcga-maf-cols is specified. This will only be in TcgaMafOutputRenderer. Requires refactoring of the header to annotation mapping paradigm.
• [x] File an issue that this prioritization is not available for VCF output. Get to it in the future, since not needed for M2. (NOT NEEDED, taken care of in --help andthrows warning if input is not TCGAMAF or SIMPLE_TSV)

Additional unit/automated tests to write:

MutationDataFactoryTest

• [x] create a mutation data from the factory with overwrite allowed and see that it is okay with two createAnnotation (each w/ different value)
• [x] create a mutation data from the factory with overwrite not allowed and see that we get DuplicateAnnotationException

RunSpecFactoryTest (if time permits)

• [x] test 1+ error conditions from -i TCGAMAF

TcgaMafOutputRendererTest

• [x] Tests for testing the priority against "INPUT" annotations. Both on and off. Simply create some mutation data objects and populate accordingly.
• [x] Test that a prepended field (in the input file) gets treated like column header with "INPUT" when re-annotating

AnnotatorTest

• [x] test that we can create a run spec that allows overwriting and attempt to do it (use a dummy datasource and a text file with an existing annotation that will overwrite an existing annotation)
• [x] same as above, but configure for failure and make sure that DuplicateAnnotationException is thrown.
• [x] Attempt to do a mini collapse ONPs from TCGA MAF workflow: VCF to TCGA MAF no ONPs, then TCGA MAF to TCGA MAF with ONPs. Check that the number of mutations is correct at both steps.

[LeeTL1220]

When ready, I am going to cut a release with this functionality, only, for GDAC (@dheiman).

The rest of the M2 requirements can be found in issue #326