pallassgj / bpipe

Automatically exported from code.google.com/p/bpipe
0 stars 1 forks source link

How do I use an input file defined when I run the pipeline but is only used after several steps? #95

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. If I run a pipeline file with three input files e.g. bpipe run 
mypipeline.pipe input1.txt input2.txt input3.txt and then in my pipeline file I 
have several commands I run using only input1.txt and input2.txt and then only 
later do I use input3.txt what is the variable that defines this third input 
file.

e.g. (simplest example of concept)

step1 = {
   exec "cat $input1 $input2 > $output"
}
// I now want to run a command using the third input text file (input3.txt). 
But what is the variable called now??
step2 = {
  exec "echo I want to see this file: $input3"
}

Bpipe.run {step1 + step2}

With the above script I get a pipeline error:

=========================================== Stage step1 
============================================

=========================================== Stage step3 
============================================
Pipeline failed! (2) 

Expected 2 or more inputs but fewer provided

What is the expected output? What do you see instead?
The expected output is a message printing "I want to see this file: input3.txt" 
but I get the above error message instead

What version of the product are you using? On what operating system?
bpipe-0.9.8.6_beta_3 on ubuntu

Please provide any additional information below.

Original issue reported on code.google.com by adamwoo...@gmail.com on 19 Mar 2014 at 9:29

GoogleCodeExporter commented 9 years ago
This looks like it could be handled by the parallel tasks "input splitting" 
features:

https://code.google.com/p/bpipe/wiki/ParallelTasks

Original comment by gdeve...@gmail.com on 11 Oct 2014 at 5:23