Issue 325 First set of OxoG changes (allowing annotation overwriting and intelligent overwriting in TCGA MAF outputs)

[LeeTL1220]

This pull request is fairly large.

This implements the easy re-annotation of TCGA MAF files (-i TCGAMAF -o TCGAMAF). Preferably, the datasources should be the same as used for the input TCGA MAF, though this is not required.

Closes #325

More specifics:

• Added new TCGAMAF input format (-i TCGAMAF). This should only be used when re-annotating a TCGA MAF. Behind-the-scenes, this just maps to -i MAFLITE --allow-overwriting --prune-tcga-maf-cols.
• Added --allow-overwriting, which tells Oncotator to allow overwriting of existing annotation values. This will make DuplicateAnnotationExceptions impossible, but allows a user to create inconsistent states. Currently, this is only supported for TCGAMAF input.
• Added --prune-tcga-maf-cols to CLI. This option is only useful for TCGAMAF output. This tells the TcgaMafOutputRenderer to take annotations from "INPUT" datasources as second choice to ones from a proper datasource. This allows the aliasing structure to stay in place while honoring overwritten annotations (see --allow-overwriting above). The default behavior of TCGA MAF is to take the annotation with the column name as highest priority. However, this causes issues when using the TCGA MAF as an input, since you will not be able to overwrite (since the datasources write to one of the column aliases).
• Updates to CLI parameter validation to accommodate new parameters described above.
• (dev) Use a MutationDataFactory for creating mutations, instead of calling a constructor directly. This was a refactor that was a long time in coming. This allows the Annotator (or other class using MutationData) to control whether a DuplicateAnnotationException is throw when trying to overwrite an annotation value. This also paves the way for more sophisticated behavior in the future. A convenience static call can also be used to create MutationDatas for unit tests, etc.
• (dev) Annotator will pass an instance of MutationDataFactory to the InputCreator to use. All InputCreators have been updated, as has the abstract class -- to require it in future implementations.
• (dev) Added FieldMapCreator for TCGA MAF output. This is a convenience class that creates a column name to annotation name mapping that is 1:1. It handles the prioritization of INPUT annotations, etc.
• (dev) Updated all unit tests to move from the MutationData constructor to calling the static MutationDataFactory.default_create(...) method.

[LeeTL1220]

@dheiman FYI... When this is reviewed and merged, I will cut a release (and update the FH module).

[dheiman]

Awesome, thanks!

On Mon, Jul 13, 2015 at 1:47 PM, Lee Lichtenstein notifications@github.com wrote:

@dheiman https://github.com/dheiman FYI... When this is reviewed and merged, I will cut a release (and update the FH module).

— Reply to this email directly or view it on GitHub https://github.com/broadinstitute/oncotator/pull/327#issuecomment-121003982 .

[LeeTL1220]

@dheiman I am pretty close to completing the next pull request (which depends on this one as well). I would like to finish that one, get it reviewed, then release. Effectively, this will probably get you your functionality at a cost of little to no time.

[lbergelson]

@LeeTL1220 I didn't get a chance to take a proper look at this yet. I'll do it first thing tomorrow.

[LeeTL1220]

@lbergelson No prob. I should have the next one almost ready to go. I just need to confirm it still works after this one is merged into it.

[lbergelson]

It looks like MutUtils.createFieldsMapping should be deleted and replaced by FieldMapCreator.create_field_map in all instances unless there is a reason to keep the other around.

[lbergelson]

@LeeTL1220 Review complete. Minor comments, looks good.

[LeeTL1220]

Removed MutUtils.createFieldMapping and updated other classes appropriately.

[LeeTL1220]

Running unit tests now.....

[LeeTL1220]

@lbergelson Changes have been pushed to address everything above, except where noted in conversation above. All unit tests pass on my laptop.