Open lxwgcool opened 3 years ago
There are many files need to be copied from biowulf to S3 in many different folders.
Root Path in S3: lix33/COVID19/UpstreamAnalysis/PostPrimaryRun/Data/10_11_2021/low_input_01_96_keytable
-------We need to copy the file below from biowulf to s3 /data/COVID_WGS/lix33/Test/2ndpipeline/Build
/data/COVID_WGS/lix33/Test/2ndpipeline/Data/BAM_reformatted/BAM_original/WGS /data/COVID_WGS/lix33/Test/2ndpipeline/Data/BAM_reformatted/BAM_recalibrated/WGS
/data/COVID_WGS/lix33/Test/2ndpipeline/Data/secondary_buf/cluster_job_logs /data/COVID_WGS/lix33/Test/2ndpipeline/Data/secondary_buf/coverage_report /data/COVID_WGS/lix33/Test/2ndpipeline/Data/secondary_buf/PRE_QC
/data/COVID_WGS/UpstreamAnalysis/PostPrimaryRun/Data/BAM/Batch/low_input/Flag /data/COVID_WGS/UpstreamAnalysis/PostPrimaryRun/Data/BAM/Batch/low_input/Log /data/COVID_WGS/UpstreamAnalysis/PostPrimaryRun/Data/BAM/Batch/low_input*/Report
/data/COVID_WGS/UpstreamAnalysis/PostPrimaryRun/Keytable/low_input
Added new function to remove the backup related data from biowulf
Notice
There are many files need to be copied from biowulf to S3 in many different folders.
Detail
Root Path in S3: lix33/COVID19/UpstreamAnalysis/PostPrimaryRun/Data/10_11_2021/low_input_01_96_keytable
-------We need to copy the file below from biowulf to s3 /data/COVID_WGS/lix33/Test/2ndpipeline/Build
/data/COVID_WGS/lix33/Test/2ndpipeline/Data/BAM_reformatted/BAM_original/WGS /data/COVID_WGS/lix33/Test/2ndpipeline/Data/BAM_reformatted/BAM_recalibrated/WGS
/data/COVID_WGS/lix33/Test/2ndpipeline/Data/secondary_buf/cluster_job_logs /data/COVID_WGS/lix33/Test/2ndpipeline/Data/secondary_buf/coverage_report /data/COVID_WGS/lix33/Test/2ndpipeline/Data/secondary_buf/PRE_QC
/data/COVID_WGS/UpstreamAnalysis/PostPrimaryRun/Data/BAM/Batch/low_input/Flag /data/COVID_WGS/UpstreamAnalysis/PostPrimaryRun/Data/BAM/Batch/low_input/Log /data/COVID_WGS/UpstreamAnalysis/PostPrimaryRun/Data/BAM/Batch/low_input*/Report
/data/COVID_WGS/UpstreamAnalysis/PostPrimaryRun/Keytable/low_input