Support for manual set of regions

[GoogleCodeExporter]

I don;t know if this feature is implemented or not:

I would like to manually support a list of regions to the pipeline.

I have the regions in a file in this format:

chr1:1-120008
chr1:611139-2614481
chr1:2888045-3035860
chr1:3052688-3210701
chr1:3251715-3882009
chr1:4422110-4533247
chr1:4665104-4893850

I want to generate N stages where each stage handle an input file (bam) and a 
region from the list.

I can make something like this in the pipeline:

def regions = new File("TruSeq_10k.intervals").text.split("\n")

But how to inject the regions in the Bpipe.run command?

Regards

Original issue reported on code.google.com by davide.r...@gmail.com on 22 Oct 2013 at 2:15

[GoogleCodeExporter]

Sorry I figure out how to do it:

def regions = new File("TruSeq_10k.intervals").text.split("\n")

hello = {
    exec """
        echo "execute_hello with $region";cp $input $output
    """
}

run {
    chr(regions) * [hello]
}

Regards

Original comment by davide.r...@gmail.com on 22 Oct 2013 at 2:18

[GoogleCodeExporter]

Further exploring I enter in this problem:

My input file have 4544 intervals ...

If I run the previous pipeline I get:

java.lang.OutOfMemoryError: unable to create new native thread

Original comment by davide.r...@gmail.com on 22 Oct 2013 at 2:36

[GoogleCodeExporter]

Still struggling with this resource problem:

even setting

export MAX_JAVA_MEM=4096M; bpipe run -v -n 10 -m 4096 region_first.groovy 
input.bam

I still getting:

java.lang.OutOfMemoryError: unable to create new native thread

In log I notice that 

 log.info("Waiting for " + runningCount.get() + " parallel stages to complete (pool.active=${pool.activeCount} pool.tasks=${pool.taskCount})" )

(Concurreny class, line 179) is never called

Original comment by davide.r...@gmail.com on 23 Oct 2013 at 9:56

[GoogleCodeExporter]

This seems related to 

    new SynchronousQueue<Runnable>()

In Concurrency.groovy constructor

If I switch to:

   new LinkedBlockingQueue<Runnable>() 

in Concurreny constructor all seems working. 

Hope it helps

Original comment by davide.r...@gmail.com on 23 Oct 2013 at 10:58

[GoogleCodeExporter]

With the change:  new LinkedBlockingQueue<Runnable>()  the java process is 
stable and generate around 50 threads on a MAC OSX 2.9 Ghz Core i7

Original comment by davide.r...@gmail.com on 23 Oct 2013 at 1:38