Closed gdevenyi closed 9 years ago
In comparison, here's running with -m 14000MB
1 bpipe.Runner [1] INFO |3:49:45 Starting
2 bpipe.Runner [1] INFO |3:49:45 OS: Linux (2.6.32-504.3.3.el6.x86_64) Java: 1.8.0_45 Vendor: Oracle Corporation
3 bpipe.Runner [1] INFO |3:49:45 Initializing plugins ...
4 bpipe.Config [1] INFO |3:49:45 No plugins directory found: /scratch/m/mchakrav/gdevenyi/bpipe-adni-test/output/.bpipe/plugins
5 bpipe.Runner [1] INFO |3:49:45 Reading user config ...
6 bpipe.Config [1] INFO |3:49:45 Reading Bpipe configuration from /scratch/m/mchakrav/gdevenyi/bpipe-adni-test/output/../minc-bpipe-library/bpipe.config
7 bpipe.Config [1] INFO |3:49:45 No local configuration file found
8 bpipe.Utils [22] INFO |3:49:45 Read config from /scratch/m/mchakrav/gdevenyi/bpipe-adni-test/output/../minc-bpipe-library/bpipe.config executed in 0.233 seconds
9 bpipe.Utils [21] INFO |3:49:45 Read config from /project/m/mchakrav/quarantine/bpipe/0.9.8.7/build/bpipe-0.9.8.7/bin/../bpipe.config executed in 0.276 seconds
10 bpipe.Config [1] INFO |3:49:45 Merging pipeline config file
11 bpipe.Utils [1] INFO |3:49:45 Read user config executed in 0.580 seconds
12 bpipe.Runner [1] INFO |3:49:45 Maximum threads specified as 8
13 bpipe.Runner [1] INFO |3:49:45 Maximum memory specified as 14000MB
14 bpipe.Runner [1] INFO |3:49:45 Loading tool database ...
15 bpipe.ToolDatabase [23] INFO |3:49:45 Loading tool database from user configuration
16 bpipe.NotificationManager [26] INFO |3:49:45 Configuring notifications
17 bpipe.Concurrency [25] INFO |3:49:45 Creating thread pool with 8 threads to execute parallel pipelines
18 bpipe.NotificationManager [26] INFO |3:49:45 Trying class name file for notification channel file
19 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool samtools
20 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool bwa
21 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool bowtie
22 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool fastqc
23 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool coverageBed
24 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool fastx_clipper
25 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool convert2annovar.pl
26 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool summarize_annovar.pl
27 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool bowtie2
28 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool GenomeAnalysisTK
29 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool MarkDuplicates
30 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool signalp
31 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool variant_effect_predictor.pl
32 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool igvtools
33 bpipe.Tool [23] INFO |3:49:45 Configured meta data about tool ls
34 bpipe.Runner [1] INFO |3:49:45 No CLI parameters specified
35 bpipe.NotificationManager [26] INFO |3:49:46 Unable to create notification channel using class file: java.lang.ClassNotFoundException: file
36 bpipe.NotificationManager [26] INFO |3:49:46 Trying class name bpipe.FILENotificationChannel for notification channel file
37 bpipe.NotificationManager [26] INFO |3:49:46 Unable to create notification channel using class bpipe.FILENotificationChannel: java.lang.ClassNotFoundException: bpipe.FILENotificationChannel
38 bpipe.NotificationManager [26] INFO |3:49:46 Trying class name bpipe.file for notification channel file
39 bpipe.NotificationManager [26] INFO |3:49:46 Unable to create notification channel using class bpipe.file: java.lang.ClassNotFoundException: bpipe.file
40 bpipe.NotificationManager [26] INFO |3:49:46 Trying class name bpipe.FileNotificationChannel for notification channel file
41 bpipe.NotificationManager [26] INFO |3:49:46 Successfully created notification channel using class bpipe.FileNotificationChannel
42 bpipe.Runner [1] INFO |3:49:46 Parsing script ...
43 bpipe.Runner [1] INFO |3:49:46 Run ...
44 bpipe.Pipeline [1] INFO |3:49:46 Evaluating library file /scratch/m/mchakrav/gdevenyi/bpipe-adni-test/minc-bpipe-library/minc-library.bpipe
45 bpipe.PipelineCategory [1] INFO |3:49:46 Found closure variable mincconvert
46 bpipe.PipelineCategory [1] INFO |3:49:46 Found closure variable n3correct
47 bpipe.PipelineCategory [1] INFO |3:49:46 Found closure variable nlm_denoise
Confirmed in #149 that command line parsing of MB and GB are both broken
Thanks - parsing issues fixed in 2c65801.
I'm calling bpipe like this:
The bpipe log file looks like:
Note the following line in the log:
Somewhere, bpipe has picked up 4000 as the memory, so my 6GB jobs never run