search

Kurt-Hetrick / CIDR_SOMATIC_WES_PIPELINES

currently for paired tumor - normal WES samples processed with TWIST UMIs
MIT License
1 stars 0 forks source link

aggregation in cloud #1

Closed Kurt-Hetrick closed 1 year ago

Kurt-Hetrick commented 2 years ago

this becomes complicated. if you want to aggregate new data for sample with previously existing data, then you have to create a separate workflow which does not redo all of the previous steps for the previously generated data because that would just be throwing money away on compute costs. sooooo, yeah. have to think about best way to go about doing this. and what to do with previously generated data. do you overwrite or generate new files in new space. if you generate new files in new space, then you have to think about deleting data in old space.

Kurt-Hetrick commented 1 year ago

cloud is not a priority/preferred anymore and in any case, it would have been the same as local. rerun/overwrite pre-existing data. seems like that was how it was done for WGS as well so I wouldn't try to the nightmare of trying to determine whether or not to overwrite pre-existing data.