search

CoBrALab / minc-bpipe-library

Library of bpipe implemented commands commonly applied to minc datafiles, suitable for importing into other pipelines
Other
13 stars 8 forks source link

Creating a Singularity recipe #32

Open shots47s opened 5 years ago

shots47s commented 5 years ago

Added initial and currently incomplete Singularity recipe based on Vladmir Fonov's Minc Tools 1.9.16 image on singularity hub. It currently installs:

  1. ANTS
  2. Java
  3. Bpipe

It has been tested up to a point where it was looking for quarantine.

gdevenyi commented 5 years ago

Thanks for the initial skeleton.

A few things to start: minc-toolkit-containers/1.9.16 already contains ANTs and minc-stuffs

After that, it looks like the container has at least the base mni models and beast-library-1.1, which is most of the QUARANTINE_PATH contents, probably just need to do some symlinking tricks to create the right paths.

shots47s commented 5 years ago

Ah, good to know, that would have made things easier. Minc-stuffs was a bit of a residual from another project.

gdevenyi commented 5 years ago

As far as installing bpipe, untarring the release tarball is quicker/easier and then we'll have a stamped version as well: https://github.com/ssadedin/bpipe/releases

shots47s commented 5 years ago

Ok, I have modified the Singularity image, when I follow the run procedure on the README.md by running:

bpipe run -n 4 /minc-bpipe-library/pipeline.bpipe ./MRI_Image.mnc

In short, it is looking for qsub. For it to run in Singularity, we should customize to run locally, as it will run in the image. So what would be the next steps then. Modifying the bpipe.config?

gdevenyi commented 5 years ago

For a containerized image, you can just delete bpipe.config

shots47s commented 5 years ago

Ok, great, I have added removing this file to the container, when I run the command now, I get a quick return. The .bpipe directory is created, and nothing seems to be happening. How should we debug now, I can send by email my logs or I can post the log here (although may clog things up).

gdevenyi commented 5 years ago

Don't mind clogging up github, its not my email inbox :)

shots47s commented 5 years ago

Ok, here is what I get in the logs: 

bpipe.Runner    [1] INFO    |9:21:01 Starting 
bpipe.Runner    [1] INFO    |9:21:01 OS: Linux (4.4.24-1.el7.elrepo.x86_64) Java: 1.8.0_03-Ubuntu Vendor: Oracle Corporation 
bpipe.Runner    [1] INFO    |9:21:01 Initializing plugins ... 
bpipe.Config    [1] INFO    |9:21:01 No plugins directory found: /home/users/sbrown/BPipe-Singularity/.bpipe/plugins 
bpipe.Runner    [1] INFO    |9:21:01 =================== GUID=0258c03b1167024cad3ca825c296577ab1f7b77a PID=27153 (27153) ============== 
bpipe.Runner    [1] INFO    |9:21:01 Reading user config ...  
bpipe.Config    [1] INFO    |9:21:01 No configuration file found in same dir as pipeline file /minc-bpipe-library/bpipe.config 
bpipe.Config    [1] INFO    |9:21:01 No local configuration file found 
bpipe.Utils [24]    INFO    |9:21:02 Read config from /bpipe/bin/../bpipe.config executed in 0.369 seconds 
bpipe.Utils [1] INFO    |9:21:02 Read user config executed in 0.616 seconds 
bpipe.Runner    [1] INFO    |9:21:02 Maximum threads specified as 4 
bpipe.Runner    [1] INFO    |9:21:02 Loading tool database ...  
bpipe.NotificationManager   [28]    INFO    |9:21:02 Configuring notifications 
bpipe.ToolDatabase  [25]    INFO    |9:21:02 Loading tool database from user configuration 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Trying class name file for notification channel file 
bpipe.Concurrency   [27]    INFO    |9:21:02 Creating thread pool with 8 threads to execute parallel pipelines 
bpipe.Dependencies  [29]    INFO    |9:21:02 No cached output graph (outputGraph2.ser) available: will be computed from property files 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool samtools 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool bwa 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool xhmm 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool ExomeDepth 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool VariantAnnotation 
bpipe.Runner    [1] INFO    |9:21:02 No CLI parameters specified 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool CODEX 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool cn.mops 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool conifer 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool pandas 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool pytables 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool matplotlib 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool seaborn 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool bowtie 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool fastqc 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool coverageBed 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool fastx_clipper 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool convert2annovar.pl 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool summarize_annovar.pl 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool bowtie2 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool GenomeAnalysisTK 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool MarkDuplicates 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool signalp 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool variant_effect_predictor.pl 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool igvtools 
bpipe.Tool  [25]    INFO    |9:21:02 Configured meta data about tool ls 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Unable to create notification channel using class file: java.lang.ClassNotFoundException: file 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Trying class name bpipe.FILENotificationChannel for notification channel file 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Unable to create notification channel using class bpipe.FILENotificationChannel: java.lang.ClassNotFoundException: bpipe.FILENotificationChannel 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Trying class name bpipe.file for notification channel file 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Unable to create notification channel using class bpipe.file: java.lang.ClassNotFoundException: bpipe.file 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Trying class name bpipe.FileNotificationChannel for notification channel file 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Successfully created notification channel using class bpipe.FileNotificationChannel 
bpipe.NotificationManager   [28]    INFO    |9:21:02 Subscribing channel [:] to event SEND 
bpipe.Runner    [1] INFO    |9:21:02 Parsing script ...  
bpipe.Runner    [1] INFO    |9:21:02 Run ...  
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable mincconvert 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n3correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable anlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correctsimple 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable dumb_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nuyl_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable pol_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable inormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable cutneckapplyautocrop 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable linear_bestlinreg 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nonlinear_antsRegistration 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extract_DBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable resample_to_lsq6_space 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastnormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastmask 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastextract 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable classify_clean 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extractvbm 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment is loading external stages 
bpipe.Pipeline  [1] WARNING |9:21:02 Pipeline folder /home/users/sbrown/bpipes could not be found 
bpipe.Pipeline  [1] INFO    |9:21:02 Evaluating paths: [] 
bpipe.Pipeline  [1] INFO    |9:21:02 Adding stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Added stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment finished loading external stages 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable mincconvert 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n3correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable anlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correctsimple 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable dumb_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nuyl_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable pol_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable inormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable cutneckapplyautocrop 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable linear_bestlinreg 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nonlinear_antsRegistration 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extract_DBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable resample_to_lsq6_space 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastnormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastmask 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastextract 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable classify_clean 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extractvbm 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beast 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment is loading external stages 
bpipe.Pipeline  [1] WARNING |9:21:02 Pipeline folder /home/users/sbrown/bpipes could not be found 
bpipe.Pipeline  [1] INFO    |9:21:02 Evaluating paths: [] 
bpipe.Pipeline  [1] INFO    |9:21:02 Adding stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Added stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment finished loading external stages 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable mincconvert 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n3correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable anlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correctsimple 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable dumb_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nuyl_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable pol_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable inormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable cutneckapplyautocrop 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable linear_bestlinreg 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nonlinear_antsRegistration 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extract_DBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable resample_to_lsq6_space 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastnormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastmask 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastextract 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable classify_clean 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extractvbm 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beast 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable VBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable defaceapply 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment is loading external stages 
bpipe.Pipeline  [1] WARNING |9:21:02 Pipeline folder /home/users/sbrown/bpipes could not be found 
bpipe.Pipeline  [1] INFO    |9:21:02 Evaluating paths: [] 
bpipe.Pipeline  [1] INFO    |9:21:02 Adding stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Added stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment finished loading external stages 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable mincconvert 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n3correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable anlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correctsimple 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable dumb_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nuyl_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable pol_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable inormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable cutneckapplyautocrop 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable linear_bestlinreg 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nonlinear_antsRegistration 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extract_DBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable resample_to_lsq6_space 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastnormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastmask 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastextract 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable classify_clean 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extractvbm 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beast 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable VBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable defaceapply 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable deface 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment is loading external stages 
bpipe.Pipeline  [1] WARNING |9:21:02 Pipeline folder /home/users/sbrown/bpipes could not be found 
bpipe.Pipeline  [1] INFO    |9:21:02 Evaluating paths: [] 
bpipe.Pipeline  [1] INFO    |9:21:02 Adding stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Added stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment finished loading external stages 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable mincconvert 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n3correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable anlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correctsimple 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable dumb_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nuyl_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable pol_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable inormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable cutneckapplyautocrop 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable linear_bestlinreg 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nonlinear_antsRegistration 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extract_DBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable resample_to_lsq6_space 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastnormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastmask 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastextract 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable classify_clean 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extractvbm 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beast 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable VBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable defaceapply 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable deface 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable cutneck 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable resample 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable clean_and_center 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable QC 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment is loading external stages 
bpipe.Pipeline  [1] WARNING |9:21:02 Pipeline folder /home/users/sbrown/bpipes could not be found 
bpipe.Pipeline  [1] INFO    |9:21:02 Evaluating paths: [] 
bpipe.Pipeline  [1] INFO    |9:21:02 Adding stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Added stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment finished loading external stages 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable mincconvert 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n3correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable anlm_denoise 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correct 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable n4correctsimple 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable dumb_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nuyl_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable pol_normalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable inormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable cutneckapplyautocrop 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable linear_bestlinreg 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable nonlinear_antsRegistration 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extract_DBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable resample_to_lsq6_space 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastnormalize 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastmask 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beastextract 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable classify_clean 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable extractvbm 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable beast 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable VBM 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable defaceapply 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable deface 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable cutneck 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable resample 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable clean_and_center 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable QC 
bpipe.PipelineCategory  [1] INFO    |9:21:02 Found closure variable preprocess 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment is loading external stages 
bpipe.Pipeline  [1] WARNING |9:21:02 Pipeline folder /home/users/sbrown/bpipes could not be found 
bpipe.Pipeline  [1] INFO    |9:21:02 Evaluating paths: [] 
bpipe.Pipeline  [1] INFO    |9:21:02 Adding stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Added stages from evaluation of [/home/users/sbrown/bpipes] 
bpipe.Pipeline  [1] INFO    |9:21:02 Segment finished loading external stages 
bpipe.ExecutorPool  [1] INFO    |9:21:02 Shutting down 0 preallocation pools 
bpipe.NotificationManager   [1] INFO    |9:21:02 Channel file does not support close 
bpipe.ExecutorPool  [22]    INFO    |9:21:02 Shutting down 0 preallocation pools
shots47s commented 5 years ago

Here is my command.log:

####################################################################################################
# Starting pipeline at Thu Dec 06 20:02:56 UTC 2018
# Input files:  ./MRI_Image.mnc
# Output Log:  .bpipe/logs/19189.log
# Stage n4correct (MRI_Image)
# ################ Finished at Thu Dec 06 20:02:57 UTC 2018 Duration = 0.812 seconds #################

####################################################################################################
# Starting pipeline at Thu Dec 06 20:03:23 UTC 2018
# Input files:  ./MRI_Image.mnc
# Output Log:  .bpipe/logs/19316.log
# Stage n4correct (MRI_Image)
iterativeN4_multispectral.sh MRI_Image.n4correct.mnc ./MRI_Image.mnc
gdevenyi commented 5 years ago

Okay, so, before going over this any further, I'm going to add a clearer dependencies list to the README. Expect it coming.

gdevenyi commented 5 years ago

Hi @shots47s I added a much clearer description of dependencies at https://github.com/CobraLab/minc-bpipe-library#installation-dependencies

Can you take a look and see how much effort it is to bring in the extra dependencies?

The only other problem with this is that pathing is a bit hard-coded right now according to the CIC quarantine structure, see https://github.com/CobraLab/minc-bpipe-library/blob/master/minc-library.bpipe#L1-L33

I have to address that with: https://github.com/CobraLab/minc-bpipe-library/issues/27

shots47s commented 5 years ago

I will work to update the recipe, but at first glance, bringing these in should not be a problem as long as the recipe has access to their locations. Hard coding the path within the container should be easy, we can just add the env variable to the recipe.

gdevenyi commented 5 years ago

I agree for a container hard-coding is fine, but in general I'd like to increase the flexibility to use the ADNI or pediatric templates (if your subjects are pediatric) and making this externally settable is ideal.

Also, in terms of non-container use, hard-coding makes things more difficult than it should be.

shots47s commented 5 years ago

Yep, completely agree with you.

gdevenyi commented 5 years ago

Hi @shots47s have you had a chance to address the dependencies updates I made?

gdevenyi commented 5 years ago

Hi,

Following up here, anything I can help with? I'm planning changing to a config-file based setup conditional on a bug being fixed at the bpipe level.

shots47s commented 5 years ago

Thanks gdevenyi, got buried under other things, I will get back to this soon. Am I holding up your changes?

gdevenyi commented 5 years ago

No, just warning you that there will be a config change coming when the bug is addressed. If it happens after the recipe I'll make sure it stays working.

ulloaa commented 3 years ago

Hello, if I understand correctly @shots47s was working on a singularity recipe for minc-bpipe-library, is this correct? If so, is the singularity recipe available somewhere? We are having lots of issues with minc-bpipe-library dependencies and would really benefit from such a recipe. Or, even better, a docker or singularity image? Thanks!

gdevenyi commented 3 years ago

I wrote my own Dockerfile here, https://github.com/CoBrALab/MAGeTDocker for another project which includes minc-bpipe-library which runs fine as a singularity image. Just open a shall in the container and run as a basic bpipe pipeline.

ulloaa commented 3 years ago

ok thanks!