search

NAL-i5K / Organism_Onboarding

A workflow to make organism onboarding pipeline easy to handle as an I/O pipeline
4 stars 1 forks source link

Move working_files directory over during movedata workflow #131

Closed mpoelchau closed 2 years ago

mpoelchau commented 2 years ago

In MoveData-workflow.cwl, we need to add steps that copy the working_files directory that we create to the other three servers. This will be similar to the steps dataTransfer-scaffold-2gmodProd, dataTransfer-scaffold-2gmodStage, dataTransfer-scaffold-2apollo

mpoelchau commented 2 years ago

@ZhiXuanLai this will be hard for you to test, so we'll need to work together on that part.

ZhiXuanLai commented 2 years ago

Hi @mpoelchau. Taking saceub as an example, I am wondering if working_files/saceub/SEUB3.0 is the directory we want to copy to the other three servers. Thank you!

mpoelchau commented 2 years ago

@ZhiXuanLai yes, that's it. First you'll need to make the directory on the destination server, though (similar to files_4_Apollo2Server/MoveData/createFolder.cwl). Then you can set up something along the lines of: 

baseCommand: [rsync]
arguments:
  - prefix: -rlvP
    position: 1
    valueFrom: $(inputs.PATH[1])/$(inputs.tree[0])/$(inputs.tree[1])/
  - position: 2
    valueFrom: $(inputs.Gmod_account):$(inputs.PATH[1])/$(inputs.tree[0])/$(inputs.tree[1])/
mpoelchau commented 2 years ago

Added in #136