↩️ return to expressiontool for renamesample

dmiller15 commented 4 years ago

Description

This PR attempts to address the problems arising with our alignment workflow in Cavatica. While there's nothing wrong with the existing code it appears some functionality of cwl cannot be reliably implemented in Cavatica for the time being. In this case, it's the outputEval of the tool bamtofastq_chomp.cwl. To temporarily get around this while SBG determines the cause, I have created a secondary "noEval" pipeline that returns the old preparerg expressiontool to the mix. This pipeline should be usable while SBG investigates the cause.

Type of change

[x] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

[x] Cavatica test: https://cavatica.sbgenomics.com/u/kfdrc-harmonization/kf-reference-pipeline/tasks/3ddfe774-9d4b-47de-b566-134c849e6de9/
[x] Tested the expression tool using an intermediate file from the Cavatica test and it produced an identical output. I can look into more testing of this but it indicates this new workflow will work both on CWLtool and Cavatica.

Test Configuration:

Environment: Cavatica
Test files: see test

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] I have checked my code and corrected any misspellings

migbro commented 4 years ago

Perhaps just point me to the task that has the result of the correct workflow to do a quick comparison please

dmiller15 commented 4 years ago

I don't know if I ran this exact workflow using the current production workflow. Closest is here: https://cavatica.sbgenomics.com/u/kfdrc-harmonization/kf-reference-pipeline/tasks/637bebd7-a771-46f1-9ba9-6b3d791e4d7c/. Same BAM but I also ran some fastqs with it.

migbro commented 4 years ago

I don't know if I ran this exact workflow using the current production workflow. Closest is here: https://cavatica.sbgenomics.com/u/kfdrc-harmonization/kf-reference-pipeline/tasks/637bebd7-a771-46f1-9ba9-6b3d791e4d7c/. Same BAM but I also ran some fastqs with it.

Um, hmmm, just want to run the most basic of metrics on the output crams to ensure you get the same results. It looks like a pretty fast run, can you run the official prod version on that one bam please?

dmiller15 commented 4 years ago

Ok running a comparison test: NoEval: https://cavatica.sbgenomics.com/u/kfdrc-harmonization/kf-reference-pipeline/tasks/d32bcc1a-93ab-45b9-9847-482643a8536a/ Eval: https://cavatica.sbgenomics.com/u/kfdrc-harmonization/kf-reference-pipeline/tasks/3c2982cc-0c6b-44d5-9a9e-d903cc7aa985/

kids-first / kf-alignment-workflow