Create a data migration process that migrates Experiment-specific data to Amazon S3 buckets. The program triggers a data transfer between the local file system and our S3 buckets.
Only data associated to a project is transferred.
Data to transfer include:
1) Path to reads: /data/scratch/rna-seq/team-name/project_dir
2) Path to pcf files: /path2pcff_iles/pipeline-runs-meta/project_dir
3) Path to json files: /path2json_files/rna-seq/project_dir"
4) Pipeline cwl script: /path2/cwl_script
5) Path to reference indexes: /data/transformed/indexer/dataset-version/organism*
6) Path to reference datasets: /data/scratch/dataSource-version/organism*
LucyHut
commented
5 years ago
Create a data migration process that migrates Experiment-specific data to Amazon S3 buckets. The program triggers a data transfer between the local file system and our S3 buckets. Only data associated to a project is transferred.
Data to transfer include: