Add Normal sample ID's to the fillout workflows

stevekm commented 2 years ago

This requires refactoring the input schema into an array of records.

stevekm commented 2 years ago

@allanbolipata the updated input schema is shown here;

https://github.com/mskcc/pluto-cwl/blob/342e6f1f4f7a3839e579fbe96ccc8d6f7a61ac77/tests/test_samples_fillout_index_workflow_cwl.py#L41-L70

stevekm commented 2 years ago

This needs some changes

Not all samples should go through maf_filter step. For our workflows, clinical samples are assumed to have been previously curated and should not be filtered again. However research samples have not been curated so should be filtered

The following things need to be changed

for each sample record input item, add a boolean flag (needs_filter) that can be used to indicate if the sample should get filtered or not
get rid of the "unindexed" designation and just re-index all bam files ; this will collapse the input schema and greatly simplify the workflow
update the test case to simulate the usage of pre-curated mutations with a small demo mutation dataset vs. a regular full sized sample maf, instead of the current method that uses multiple full-size mafs

@allanbolipata @timosong

stevekm commented 2 years ago

also I think we should be able to include bam_file in the sample records instead of having it as a separate input array

stevekm commented 2 years ago

also I think we should be able to include bam_file in the sample records instead of having it as a separate input array

forget about this for now, I had forgotten that there are still issues with CWL version 1.0 and array record entries having secondary files

mskcc / pluto-cwl

Add Normal sample ID's to the fillout workflows #95