Open ByroneCole-SageBionetworks opened 1 year ago
We reached out to CHOP (Eric, Allison, Yuankun) to answer questions we had about the CHOP BixOps. Here is the link to the slack thread: https://teaminclude.slack.com/archives/C03K4BHD3QC/p1673973750582959
On the datahops call on 3/23/2023, we stated that Sage demonstrated the ability to launch and monitor workflows on Cavatica using Orca so we are calling this particular portion done.
The rest of the bullet points are BONUS, but we will continue to try to tackle them.
For the V3 tech plan, I am including a draft of the automation of data processing SOP for V4 and the future of this project.
The genomic data processing for INCLUDE occurs within the Cavatica platform developed and maintained by Velsera. The CHOP team have developed many bioinformatics operation pipelines that capture all the steps prior and post processing on Cavatica. The Sage Bionetworks team is responsible for learning and attempting to automate this process as much as possible. The bulk of the work is in learning the BixOps and then attempting to automate all the steps.
This is a high level flow chart for the steps required to leverage Orca for genomic data processing in INCLUDE. (Lucid link)
On a high level:
Depending on the complexity and completeness of any BixOps workflow, we estimate that it will take around a quarter to fully automate a workflow, if deemed feasible.
Only comment, think about capacity... given our funding, how many "Execute Orca recipe on entire dataset" can we do in a given quarter?
Other thing... think about adding a flow step for "trigger notification of CHOP"... and adding status back to DFA as that's deployed for INCLUDE
[ ] CHOP RNASeq BixOps Automation. JIRA
[ ] BONUS: CHOP WGS BixOps Automation. JIRA