RNA-seq --aligner not working - error

[dshechter]

Just installed cipher, looks really neat and exactly what my group needs for straightforward analysis. However, I just tried running it for the first time on a GEO dataset and encountered this odd error, even though I set the --alinger in the ommand to star (and tried others, non worked):

nextflow run /opt/main.nf --mode rna --config ./path/to/cfog --fasta ./ --gtf /path/to/gtf --lib s --readLen 50 --strandInfo frFirstStrand --expInfo ./DMSO_exp_config.txt --aligner star --threads 20 N E X T F L O W ~ version 0.25.1 Launching /opt/main.nf [small_easley] - revision: b095c773ef WARN: Access to undefined parameter analysis -- Initialise it to a default value eg. params.analysis = some_value ERROR ~ --aligner should be one of the following: bbmap, bowtie2, bwa, hisat2, star

-- Check '.nextflow.log' file for details

Nothing helpful in the logs. Any idea? Why isn't it recognizing the flag chosing star?

[c-guzman]

Looking into this.

Does it work if you switch over to HISAT2?

On Nov 21, 2017 1:31 AM, "dshechter" notifications@github.com wrote:

Just installed cipher, looks really neat and exactly what my group needs for straightforward analysis. However, I just tried running it for the first time on a GEO dataset and encountered this odd error, even though I set the --alinger in the ommand to star (and tried others, non worked):

nextflow run /opt/main.nf --mode rna --config ./path/to/cfog --fasta ./ --gtf /path/to/gtf --lib s --readLen 50 --strandInfo frFirstStrand --expInfo ./DMSO_exp_config.txt --aligner star --threads 20 N E X T F L O W ~ version 0.25.1 Launching /opt/main.nf [small_easley] - revision: b095c773ef WARN: Access to undefined parameter analysis -- Initialise it to a default value eg. params.analysis = some_value ERROR ~ --aligner should be one of the following: bbmap, bowtie2, bwa, hisat2, star

-- Check '.nextflow.log' file for details

Nothing helpful in the logs. Any idea? Why isn't it recognizing the flag chosing star?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fwp4PPwam6lakLCxR2XFwtQ8O12QVks5s4lFmgaJpZM4QlRIu .

[c-guzman]

Nevermind, just re-read your post. Very odd. Will get back to you today!

On Nov 21, 2017 1:31 AM, "dshechter" notifications@github.com wrote:

Just installed cipher, looks really neat and exactly what my group needs for straightforward analysis. However, I just tried running it for the first time on a GEO dataset and encountered this odd error, even though I set the --alinger in the ommand to star (and tried others, non worked):

nextflow run /opt/main.nf --mode rna --config ./path/to/cfog --fasta ./ --gtf /path/to/gtf --lib s --readLen 50 --strandInfo frFirstStrand --expInfo ./DMSO_exp_config.txt --aligner star --threads 20 N E X T F L O W ~ version 0.25.1 Launching /opt/main.nf [small_easley] - revision: b095c773ef WARN: Access to undefined parameter analysis -- Initialise it to a default value eg. params.analysis = some_value ERROR ~ --aligner should be one of the following: bbmap, bowtie2, bwa, hisat2, star

-- Check '.nextflow.log' file for details

Nothing helpful in the logs. Any idea? Why isn't it recognizing the flag chosing star?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fwp4PPwam6lakLCxR2XFwtQ8O12QVks5s4lFmgaJpZM4QlRIu .

[dshechter]

thanks, it's very odd. Happened with the default (no flag) or with any other aligner (including hisat). Here is the .nextflow.log from one I just tried:

Nov-21 12:46:06.365 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 0.25.1 Nov-21 12:46:06.395 [main] INFO nextflow.cli.CmdRun - Launching /opt/main.nf [small_ptolemy] - revision: b095c773ef Nov-21 12:46:06.683 [main] DEBUG nextflow.Session - Session uuid: b3f3ddb2-e703-4a46-a105-6186d2cef565 Nov-21 12:46:06.683 [main] DEBUG nextflow.Session - Run name: small_ptolemy Nov-21 12:46:06.685 [main] DEBUG nextflow.Session - Executor pool size: 40 Nov-21 12:46:06.703 [main] DEBUG nextflow.cli.CmdRun - Version: 0.25.1 build 4460 Modified: 27-06-2017 12:31 UTC (08:31 EDT) System: Linux 3.10.0-514.26.2.el7.x86_64 Runtime: Groovy 2.4.11 on OpenJDK 64-Bit Server VM 1.8.0_121-b15 Encoding: UTF-8 (UTF-8) Process: 21685@ CPUs: 40 - Mem: 125.6 GB (2.4 GB) - Swap: 6 GB (6 GB) Nov-21 12:46:06.740 [main] DEBUG nextflow.Session - Work-dir: /mnt/raid/NGS/NGS_Data/GEO/x [ext2/ext3] Nov-21 12:46:06.741 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /opt/bin Nov-21 12:46:06.912 [main] DEBUG nextflow.Session - Session start invoked Nov-21 12:46:06.922 [main] DEBUG nextflow.processor.TaskDispatcher - Dispatcher > start Nov-21 12:46:06.923 [main] DEBUG nextflow.script.ScriptRunner - > Script parsing Nov-21 12:46:09.055 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution Nov-21 12:46:09.071 [main] WARN nextflow.script.ScriptBinding - Access to undefined parameter analysis -- Initialise it to a default value eg. params.analysis = some_value Nov-21 12:46:09.074 [main] ERROR nextflow.Nextflow - --aligner should be one of the following: bbmap, bowtie2, bwa, hisat2, star

[c-guzman]

Did you rename the nextflow script from cipher.nf to main.nf as shown in your command line?

[dshechter]

No - strangely enough that was how it appeared when I cloned the repository. I just deleted it and recloned it and put it in my home dir in case there were some permission issues. FYI now its called cipher.nf.

But now it's failing on the fasta file input, which looks fine to me in my command: --fasta /data/NGS/NGS_Data/Stds/hs/GRCh38.fa. It's on a mounted RAID array. Could there be some problems with permissions or something else weird?

Here's the new error: WARN: Access to undefined parameter fa -- Initialise it to a default value eg. params.fa = some_value ERROR ~ Please specify a FASTA file.

[c-guzman]

Most likely some permission issues, I've run into this problem a few times on a couple of clusters since moving for my PhD. This has happened with reference files and even fastq files. Try creating a symlink to your fasta file on your home directory. If you get a new error (probably the GTF or something) then this is a permissions error.

On Tue, Nov 21, 2017 at 10:14 AM, dshechter notifications@github.com wrote:

No - strangely enough that was how it appeared when I cloned the repository. I just deleted it and recloned it and put it in my home dir in case there were some permission issues. FYI now its called cipher.nf.

But now it's failing on the fasta file input, which looks fine to me in my command: --fasta /data/NGS/NGS_Data/Stds/hs/GRCh38.fa. It's on a mounted RAID array. Could there be some problems with permissions or something else weird?

Here's the new error: WARN: Access to undefined parameter fa -- Initialise it to a default value eg. params.fa = some_value ERROR ~ Please specify a FASTA file.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10#issuecomment-346114162, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fws0i3_Fh3RhDePv9JrbTN_zKcqmPks5s4xLxgaJpZM4QlRIu .

-- Carlos Guzman - Graduate Student Bioinformatics & Systems Biology University of California, San Diego

*Phone*: (956) 437-6263 E-mail: cag104@ucsd.edu

[c-guzman]

Actually, if that doesn't work you may want to copy the entire fasta file to your home directory for good measure, just to make sure, I don't know how symlink handles permissions.

On Tue, Nov 21, 2017 at 10:38 AM, Carlos Guzman cag104@eng.ucsd.edu wrote:

Most likely some permission issues, I've run into this problem a few times on a couple of clusters since moving for my PhD. This has happened with reference files and even fastq files. Try creating a symlink to your fasta file on your home directory. If you get a new error (probably the GTF or something) then this is a permissions error.

On Tue, Nov 21, 2017 at 10:14 AM, dshechter notifications@github.com wrote:

No - strangely enough that was how it appeared when I cloned the repository. I just deleted it and recloned it and put it in my home dir in case there were some permission issues. FYI now its called cipher.nf.

But now it's failing on the fasta file input, which looks fine to me in my command: --fasta /data/NGS/NGS_Data/Stds/hs/GRCh38.fa. It's on a mounted RAID array. Could there be some problems with permissions or something else weird?

Here's the new error: WARN: Access to undefined parameter fa -- Initialise it to a default value eg. params.fa = some_value ERROR ~ Please specify a FASTA file.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10#issuecomment-346114162, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fws0i3_Fh3RhDePv9JrbTN_zKcqmPks5s4xLxgaJpZM4QlRIu .

-- Carlos Guzman - Graduate Student Bioinformatics & Systems Biology University of California, San Diego

*Phone*: (956) 437-6263 E-mail: cag104@ucsd.edu

-- Carlos Guzman - Graduate Student Bioinformatics & Systems Biology University of California, San Diego

*Phone*: (956) 437-6263 E-mail: cag104@ucsd.edu

[dshechter]

I moved everything (data, reference files, cipher directory) to my home directory and I still had an error ('Please specify a FASTA file'). Here are the permissions on the gtf and fa files, within my home directory:

-rwxr-xr-x 1 david shechterlab 1.4G Mar 6 2017 GRCh38.83.gtf -rwxr-xr-x 1 david shechterlab 3.0G Mar 6 2017 GRCh38.fa

Any other ideas? I'd really like to use your cipher package, and it would make it easier for my lab members. But no idea how to go forward. Thanks

[c-guzman]

Oh I see what may be going on. The newer version of CIPHER uses --fa instead of --fasta. Please try running your code as follows:

nextflow run /opt/cipher.nf --mode rna --config --fa --gtf

--lib s --readLen 50 --strandInfo frFirstStrand --expInfo --aligner star --threads 20 --subsample --downstream_analysis false --bamcoverage false This will give you a minimum testing pass which should let us make sure that the core functionality of CIPHER is working. If this works then let me know and we can continue downstream analysis. (Please note that I'm currently fixing a bug with bamcoverage so for now always run your command with --bamcoverage false until later today when I push out the fix). On Tue, Nov 21, 2017 at 11:58 AM, dshechter wrote: > I moved everything (data, reference files, cipher directory) to my home > directory and I still had an error ('Please specify a FASTA file'). Here > are the permissions on the gtf and fa files, within my home directory: > > -rwxr-xr-x 1 david shechterlab 1.4G Mar 6 2017 GRCh38.83.gtf > -rwxr-xr-x 1 david shechterlab 3.0G Mar 6 2017 GRCh38.fa > > Any other ideas? I'd really like to use your cipher package, and it would > make it easier for my lab members. But no idea how to go forward. Thanks > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > , > or mute the thread > > . > -- *Carlos Guzman - Graduate Student* *Bioinformatics & Systems Biology* University of California, San Diego *Phone**:* (956) 437-6263 *E-mail*: cag104@ucsd.edu

[dshechter]

Thanks - I figured that out myself a few minutes ago after checking the downloaded helpfile. It launched but ended with another error (regardless of directory or running as user or sudo):

Launching /home/david/cipher-workflow-platform/cipher.nf [desperate_austin] - revision: d4ab4cbdac

             RUN INFORMATION

===============================================

mode: rna config: /data/NGS/NGS_Data/GEO/X fasta: /data/NGS/NGS_Data/Stds/hs/X gtf: /data/NGS/NGS_Data/Stds/hs/X library: s read-length: 50

subsampled: false

clumpify: true bbduk: true fastqc: true multiqc: true mapping: true bamcoverage: true downstream_analysis true

threads: 5

aligner: star

ERROR ~ 3

-- Check script 'cipher.nf' at line: 322 or see '.nextflow.log' file for more details [warm up] executor > local ERROR ~ Channel fastqs_s has been used twice as an input by process clumpify and another operator

-- Check script 'cipher.nf' at line: 554 or see '.nextflow.log' file for more details

[c-guzman]

Well this is progress I would say lol. Try adding "--clumpify false" to your command for now. Might as well catch all the errors and fix them at once.

Thanks for this! It's been a while since I worked on RNA seq data.

On Nov 21, 2017 12:39 PM, "dshechter" notifications@github.com wrote:

Thanks - I figured that out myself a few minutes ago after checking the downloaded helpfile. It launched but ended with another error (regardless of directory or running as user or sudo):

Launching /home/david/cipher-workflow-platform/cipher.nf [desperate_austin] - revision: d4ab4cbdac

         RUN INFORMATION

===============================================

mode: rna config: /data/NGS/NGS_Data/GEO/X fasta: /data/NGS/NGS_Data/Stds/hs/X gtf: /data/NGS/NGS_Data/Stds/hs/X library: s read-length: 50

subsampled: false

clumpify: true bbduk: true fastqc: true multiqc: true mapping: true bamcoverage: true downstream_analysis true

threads: 5 aligner: star

ERROR ~ 3

-- Check script 'cipher.nf' at line: 322 or see '.nextflow.log' file for more details [warm up] executor > local ERROR ~ Channel fastqs_s has been used twice as an input by process clumpify and another operator

-- Check script 'cipher.nf' at line: 554 or see '.nextflow.log' file for more details

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10#issuecomment-346153688, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fwkRIdASFo3BCpUBmw-Go2mlTU6-Bks5s4zTtgaJpZM4QlRIu .

[dshechter]

Glad to help! Would love to have it working. Maybe I'll try a chip-seq dataset as a test also.

--clumpify false didn't help... same error. The runtime output is identical to before. The logfile is pretty big, not sure if I should post it here.

[c-guzman]

Go ahead and post it, or add it as an attachment. Could I also get the link to GEO dataset? The script is working fine on some RNA-seq I took from our cluster, but it's pair-ended.

Thanks!

On Tue, Nov 21, 2017 at 12:47 PM, dshechter notifications@github.com wrote:

Glad to help! Would love to have it working. Maybe I'll try a chip-seq dataset as a test also.

--clumpify false didn't help... same error. The runtime output is identical to before. The logfile is pretty big, not sure if I should post it here.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10#issuecomment-346155973, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fwng8apdl2tWWCMvqk__hq5Lboz7nks5s4zb5gaJpZM4QlRIu .

-- Carlos Guzman - Graduate Student Bioinformatics & Systems Biology University of California, San Diego

*Phone*: (956) 437-6263 E-mail: cag104@ucsd.edu

[dshechter]

GSE85107

Here is my cfg file showing the runs I'm mapping:

AML2_DMSO AML2_DMSO_rep1 /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/SRR3994142.fastq.gz AML2_DMSO AML2_DMSO_rep2 /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/SRR3994143.fastq.gz AML2_DMSO AML2_DMSO_rep3 /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/SRR3994144.fastq.gz AML3_DMSO AML3_DMSO_rep1 /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/SRR3994136.fastq.gz AML3_DMSO AML3_DMSO_rep2 /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/SRR3994137.fastq.gz AML3_DMSO AML3_DMSO_rep3 /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/SRR3994138.fastq.gz

here's the log:

[david@drdisorder fastq]$ more .nextflow.log Nov-21 15:31:00.712 [main] DEBUG nextflow.cli.Launcher - $> /home/david/anaconda2/bin/nextflow run /home/david/cipher-workflow-platform/cipher.nf --mode rna --config /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/AML2_AML3_DMSO-cipher.cfg --fa /data/NGS /NGS_Data/Stds/hs/GRch38.fa --gtf /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf --lib s --readLen 50 --strandInfo frFirstStrand --expInfo /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/DMSO_exp_config.txt --aligner star --threads 5 Nov-21 15:31:00.810 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 0.25.1 Nov-21 15:31:00.834 [main] INFO nextflow.cli.CmdRun - Launching /home/david/cipher-workflow-platform/cipher.nf [desperate_austin] - revision: d4ab4cbdac Nov-21 15:31:00.844 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/david/cipher-workflow-platform/nextflow.config Nov-21 15:31:00.845 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/david/cipher-workflow-platform/nextflow.config Nov-21 15:31:01.098 [main] DEBUG nextflow.config.ConfigBuilder - Setting config profile: 'standard' Nov-21 15:31:01.160 [main] DEBUG nextflow.Session - Session uuid: ac5c1ccb-ecfb-4129-b00a-ad3d3f6fcee6 Nov-21 15:31:01.160 [main] DEBUG nextflow.Session - Run name: desperate_austin Nov-21 15:31:01.162 [main] DEBUG nextflow.Session - Executor pool size: 40 Nov-21 15:31:01.178 [main] DEBUG nextflow.cli.CmdRun - Version: 0.25.1 build 4460 Modified: 27-06-2017 12:31 UTC (08:31 EDT) System: Linux 3.10.0-514.26.2.el7.x86_64 Runtime: Groovy 2.4.11 on OpenJDK 64-Bit Server VM 1.8.0_151-b12 Encoding: UTF-8 (UTF-8) Process: 5144@drdisorder [10.49.65.147] CPUs: 40 - Mem: 125.6 GB (756.9 MB) - Swap: 6 GB (6 GB) Nov-21 15:31:01.209 [main] DEBUG nextflow.Session - Work-dir: /mnt/raid/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/work [ext2/ext3] Nov-21 15:31:01.356 [main] DEBUG nextflow.Session - Session start invoked Nov-21 15:31:01.359 [main] DEBUG nextflow.processor.TaskDispatcher - Dispatcher > start Nov-21 15:31:01.360 [main] DEBUG nextflow.script.ScriptRunner - > Script parsing Nov-21 15:31:03.016 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution Nov-21 15:31:03.044 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.044 [main] INFO nextflow.Nextflow - RUN INFORMATION Nov-21 15:31:03.044 [main] INFO nextflow.Nextflow - =============================================== Nov-21 15:31:03.044 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - mode: rna Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - config: /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/AML2_AML3_DMSO-cipher.cfg Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - fasta: /data/NGS/NGS_Data/Stds/hs/GRch38.fa Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - gtf: /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - library: s Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - read-length: 50 Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - subsampled: false Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - clumpify: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - bbduk: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - fastqc: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - multiqc: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - mapping: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - bamcoverage: true Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - downstream_analysis true Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - threads: 5 Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - aligner: star Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - =============================================== Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.118 [Actor Thread 1] ERROR n.extension.DataflowExtensions - @unknown java.lang.ArrayIndexOutOfBoundsException: 3 at org.codehaus.groovy.runtime.dgmimpl.arrays.ObjectArrayGetAtMetaMethod.invoke(ObjectArrayGetAtMetaMethod.java:41) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.invoke(PojoMetaMethodSite.java:51) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at _nf_script_cc617d25$_run_closure1.doCall(_nf_script_cc617d25:322) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at nextflow.extension.MapOp$_apply_closure1.doCall(MapOp.groovy:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.Closure.call(Closure.java:414) at groovyx.gpars.dataflow.operator.DataflowOperatorActor.startTask(DataflowOperatorActor.java:120) at groovyx.gpars.dataflow.operator.DataflowOperatorActor.onMessage(DataflowOperatorActor.java:108) at groovyx.gpars.actor.impl.SDAClosure$1.call(SDAClosure.java:43) at groovyx.gpars.actor.AbstractLoopingActor.runEnhancedWithoutRepliesOnMessages(AbstractLoopingActor.java:293) at groovyx.gpars.actor.AbstractLoopingActor.access$400(AbstractLoopingActor.java:30) at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage(AbstractLoopingActor.java:93) at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Nov-21 15:31:03.133 [Actor Thread 1] DEBUG nextflow.Session - Session aborted -- Cause: 3 Nov-21 15:31:03.164 [Actor Thread 1] DEBUG nextflow.Session - The following nodes are still active: [operator] map

Nov-21 15:31:03.165 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null Nov-21 15:31:03.165 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local' Nov-21 15:31:03.171 [main] DEBUG nextflow.executor.Executor - Initializing executor: local Nov-21 15:31:03.172 [main] INFO nextflow.executor.Executor - [warm up] executor > local Nov-21 15:31:03.177 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=40; memory=125.6 GB; capacity=40; pollInterval=100ms; dumpInterval=5m Nov-21 15:31:03.181 [main] DEBUG nextflow.processor.TaskDispatcher - Starting monitor: LocalPollingMonitor Nov-21 15:31:03.182 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local) Nov-21 15:31:03.184 [main] DEBUG nextflow.executor.Executor - Invoke register for executor: local Nov-21 15:31:03.189 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) Nov-21 15:31:03.215 [main] DEBUG nextflow.Session - >>> barrier register (process: fetch_chrom_sizes) Nov-21 15:31:03.226 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > fetch_chrom_sizes -- maxForks: 40 Nov-21 15:31:03.252 [Actor Thread 1] DEBUG nextflow.processor.TaskProcessor - Poison pill arrived; port: 1 Nov-21 15:31:03.253 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null Nov-21 15:31:03.253 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local' Nov-21 15:31:03.254 [main] DEBUG nextflow.executor.Executor - Initializing executor: local Nov-21 15:31:03.255 [Actor Thread 2] DEBUG nextflow.processor.StateObj - State before poison: StateObj[submitted: 1; completed: 0; poisoned: false ] Nov-21 15:31:03.255 [main] DEBUG nextflow.Session - >>> barrier register (process: create_mapping_index) Nov-21 15:31:03.263 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > create_mapping_index -- maxForks: 40 Nov-21 15:31:03.268 [Actor Thread 4] DEBUG nextflow.processor.TaskProcessor - Poison pill arrived; port: 2 Nov-21 15:31:03.268 [Actor Thread 2] DEBUG nextflow.processor.StateObj - State before poison: StateObj[submitted: 1; completed: 0; poisoned: false ] Nov-21 15:31:03.297 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null Nov-21 15:31:03.298 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local' Nov-21 15:31:03.298 [main] DEBUG nextflow.executor.Executor - Initializing executor: local Nov-21 15:31:03.300 [main] DEBUG nextflow.Session - >>> barrier register (process: clumpify) Nov-21 15:31:03.310 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > clumpify -- maxForks: 40 Nov-21 15:31:03.323 [Actor Thread 3] DEBUG nextflow.util.CacheHelper - Unable to hash file: /data/NGS/NGS_Data/Stds/hs/GRch38.fa -- Cause: java.nio.file.NoSuchFileException: /data/NGS/NGS_Data/Stds/hs/GRch38.fa Nov-21 15:31:03.439 [main] ERROR nextflow.cli.Launcher - @unknown nextflow.dag.MultipleInputChannelException: Channel fastqs_s has been used twice as an input by process clumpify and another operator at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83) at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:263) at nextflow.dag.DAG.inbound(DAG.groovy:192) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:384) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:69) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:174) at nextflow.dag.DAG$_addVertex_closure1.doCall(DAG.groovy:131) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.Closure.call(Closure.java:414) at groovy.lang.Closure.call(Closure.java:430) at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2040) at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2025) at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2066) at org.codehaus.groovy.runtime.dgm$162.invoke(Unknown Source) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callSafe(AbstractCallSite.java:87) at nextflow.dag.DAG.addVertex(DAG.groovy:130) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at nextflow.dag.DAG.addProcessNode(DAG.groovy:89) at nextflow.dag.NodeMarker.addProcessNode(NodeMarker.groovy:33) at nextflow.dag.NodeMarker$addProcessNode$2.call(Unknown Source) at nextflow.processor.TaskProcessor.createOperator(TaskProcessor.groovy:484) at nextflow.processor.TaskProcessor.run(TaskProcessor.groovy:378) at nextflow.processor.TaskProcessor$run.call(Unknown Source) at nextflow.script.BaseScript.process(BaseScript.groovy:134) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:174) at _nf_script_cc617d25.run(_nf_script_cc617d25:554) at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:325) at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:156) at nextflow.cli.CmdRun.run(CmdRun.groovy:223) at nextflow.cli.Launcher.run(Launcher.groovy:410) at nextflow.cli.Launcher.main(Launcher.groovy:564) Nov-21 15:31:03.440 [Actor Thread 4] DEBUG nextflow.processor.TaskProcessor - After stop

[c-guzman]

Please give it a try now with the command I had previously sent.

On Tue, Nov 21, 2017 at 1:32 PM, dshechter notifications@github.com wrote:

GSE85107

Here is my cfg file showing the runs I'm mapping:

AML2_DMSO AML2_DMSO_rep1 /data/NGS/NGS_Data/GEO/ GSE85107_Armstrong2016_AML/fastq/SRR3994142.fastq.gz AML2_DMSO AML2_DMSO_rep2 /data/NGS/NGS_Data/GEO/ GSE85107_Armstrong2016_AML/fastq/SRR3994143.fastq.gz AML2_DMSO AML2_DMSO_rep3 /data/NGS/NGS_Data/GEO/ GSE85107_Armstrong2016_AML/fastq/SRR3994144.fastq.gz AML3_DMSO AML3_DMSO_rep1 /data/NGS/NGS_Data/GEO/ GSE85107_Armstrong2016_AML/fastq/SRR3994136.fastq.gz AML3_DMSO AML3_DMSO_rep2 /data/NGS/NGS_Data/GEO/ GSE85107_Armstrong2016_AML/fastq/SRR3994137.fastq.gz AML3_DMSO AML3_DMSO_rep3 /data/NGS/NGS_Data/GEO/ GSE85107_Armstrong2016_AML/fastq/SRR3994138.fastq.gz

here's the log:

[david@drdisorder fastq]$ more .nextflow.log Nov-21 15:31:00.712 [main] DEBUG nextflow.cli.Launcher - $> /home/david/anaconda2/bin/nextflow run /home/david/cipher-workflow- platform/cipher.nf --mode rna --config /data/NGS/NGS_Data/GEO/ GSE85107_Armstrong2016_AML/fastq/AML2_AML3_DMSO-cipher.cfg --fa /data/NGS /NGS_Data/Stds/hs/GRch38.fa --gtf /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf --lib s --readLen 50 --strandInfo frFirstStrand --expInfo /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/DMSO_exp_config.txt --aligner star --threads 5 Nov-21 15:31:00.810 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 0.25.1 Nov-21 15:31:00.834 [main] INFO nextflow.cli.CmdRun - Launching /home/david/cipher-workflow-platform/cipher.nf [desperate_austin] - revision: d4ab4cbdac Nov-21 15:31:00.844 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/david/cipher-workflow-platform/nextflow.config Nov-21 15:31:00.845 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/david/cipher-workflow-platform/nextflow.config Nov-21 15:31:01.098 [main] DEBUG nextflow.config.ConfigBuilder - Setting config profile: 'standard' Nov-21 15:31:01.160 [main] DEBUG nextflow.Session - Session uuid: ac5c1ccb-ecfb-4129-b00a-ad3d3f6fcee6 Nov-21 15:31:01.160 [main] DEBUG nextflow.Session - Run name: desperate_austin Nov-21 15:31:01.162 [main] DEBUG nextflow.Session - Executor pool size: 40 Nov-21 15:31:01.178 [main] DEBUG nextflow.cli.CmdRun - Version: 0.25.1 build 4460 Modified: 27-06-2017 12:31 UTC (08:31 EDT) System: Linux 3.10.0-514.26.2.el7.x86_64 Runtime: Groovy 2.4.11 on OpenJDK 64-Bit Server VM 1.8.0_151-b12 Encoding: UTF-8 (UTF-8) Process: 5144@drdisorder [10.49.65.147] CPUs: 40 - Mem: 125.6 GB (756.9 MB) - Swap: 6 GB (6 GB) Nov-21 15:31:01.209 [main] DEBUG nextflow.Session - Work-dir: /mnt/raid/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/work [ext2/ext3] Nov-21 15:31:01.356 [main] DEBUG nextflow.Session - Session start invoked Nov-21 15:31:01.359 [main] DEBUG nextflow.processor.TaskDispatcher - Dispatcher > start Nov-21 15:31:01.360 [main] DEBUG nextflow.script.ScriptRunner - > Script parsing Nov-21 15:31:03.016 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution Nov-21 15:31:03.044 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.044 [main] INFO nextflow.Nextflow - RUN INFORMATION Nov-21 15:31:03.044 [main] INFO nextflow.Nextflow -

Nov-21 15:31:03.044 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - mode: rna Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - config: /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/ fastq/AML2_AML3_DMSO-cipher.cfg Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - fasta: /data/NGS/NGS_Data/Stds/hs/GRch38.fa Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - gtf: /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - library: s Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - read-length: 50 Nov-21 15:31:03.045 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - subsampled: false Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - clumpify: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - bbduk: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - fastqc: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - multiqc: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - mapping: true Nov-21 15:31:03.046 [main] INFO nextflow.Nextflow - bamcoverage: true Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - downstream_analysis true Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - threads: 5 Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - aligner: star Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow -

Nov-21 15:31:03.047 [main] INFO nextflow.Nextflow - Nov-21 15:31:03.118 [Actor Thread 1] ERROR n.extension.DataflowExtensions

• @Unknown https://github.com/unknown java.lang.ArrayIndexOutOfBoundsException: 3 at org.codehaus.groovy.runtime.dgmimpl.arrays.ObjectArrayGetAtMetaMethod. invoke(ObjectArrayGetAtMetaMethod.java:41) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite. invoke(PojoMetaMethodSite.java:51) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite. call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.CallSiteArray. defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. call(AbstractCallSite.java:125) at _nf_script_cc617d25$_run_closure1.doCall(_nf_script_cc617d25:322) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod. invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass. invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite. call(PogoMetaClassSite.java:42) at org.codehaus.groovy.runtime.callsite.CallSiteArray. defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. call(AbstractCallSite.java:125) at nextflow.extension.MapOp$_apply_closure1.doCall(MapOp.groovy:50) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod. invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass. invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.Closure.call(Closure.java:414) at groovyx.gpars.dataflow.operator.DataflowOperatorActor.startTask( DataflowOperatorActor.java:120) at groovyx.gpars.dataflow.operator.DataflowOperatorActor.onMessage( DataflowOperatorActor.java:108) at groovyx.gpars.actor.impl.SDAClosure$1.call(SDAClosure.java:43) at groovyx.gpars.actor.AbstractLoopingActor.runEnhancedWithoutRepliesOnMes sages(AbstractLoopingActor.java:293) at groovyx.gpars.actor.AbstractLoopingActor.access$ 400(AbstractLoopingActor.java:30) at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage( AbstractLoopingActor.java:93) at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132) at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run( ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Nov-21 15:31:03.133 [Actor Thread 1] DEBUG nextflow.Session - Session aborted -- Cause: 3 Nov-21 15:31:03.164 [Actor Thread 1] DEBUG nextflow.Session - The following nodes are still active: [operator] map

Nov-21 15:31:03.165 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null Nov-21 15:31:03.165 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local' Nov-21 15:31:03.171 [main] DEBUG nextflow.executor.Executor - Initializing executor: local Nov-21 15:31:03.172 [main] INFO nextflow.executor.Executor - [warm up] executor > local Nov-21 15:31:03.177 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=40; memory=125.6 GB; capacity=40; pollInterval=100ms; dumpInterval=5m Nov-21 15:31:03.181 [main] DEBUG nextflow.processor.TaskDispatcher - Starting monitor: LocalPollingMonitor Nov-21 15:31:03.182 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local) Nov-21 15:31:03.184 [main] DEBUG nextflow.executor.Executor - Invoke register for executor: local Nov-21 15:31:03.189 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) Nov-21 15:31:03.215 [main] DEBUG nextflow.Session - >>> barrier register (process: fetch_chrom_sizes) Nov-21 15:31:03.226 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > fetch_chrom_sizes -- maxForks: 40 Nov-21 15:31:03.252 [Actor Thread 1] DEBUG nextflow.processor.TaskProcessor

• Poison pill arrived; port: 1 Nov-21 15:31:03.253 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null Nov-21 15:31:03.253 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local' Nov-21 15:31:03.254 [main] DEBUG nextflow.executor.Executor - Initializing executor: local Nov-21 15:31:03.255 [Actor Thread 2] DEBUG nextflow.processor.StateObj - State before poison: StateObj[submitted: 1; completed: 0; poisoned: false ] Nov-21 15:31:03.255 [main] DEBUG nextflow.Session - >>> barrier register (process: create_mapping_index) Nov-21 15:31:03.263 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > create_mapping_index -- maxForks: 40 Nov-21 15:31:03.268 [Actor Thread 4] DEBUG nextflow.processor.TaskProcessor
• Poison pill arrived; port: 2 Nov-21 15:31:03.268 [Actor Thread 2] DEBUG nextflow.processor.StateObj - State before poison: StateObj[submitted: 1; completed: 0; poisoned: false ] Nov-21 15:31:03.297 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: null Nov-21 15:31:03.298 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local' Nov-21 15:31:03.298 [main] DEBUG nextflow.executor.Executor - Initializing executor: local Nov-21 15:31:03.300 [main] DEBUG nextflow.Session - >>> barrier register (process: clumpify) Nov-21 15:31:03.310 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > clumpify -- maxForks: 40 Nov-21 15:31:03.323 [Actor Thread 3] DEBUG nextflow.util.CacheHelper - Unable to hash file: /data/NGS/NGS_Data/Stds/hs/GRch38.fa -- Cause: java.nio.file.NoSuchFileException: /data/NGS/NGS_Data/Stds/hs/GRch38.fa Nov-21 15:31:03.439 [main] ERROR nextflow.cli.Launcher - @Unknown nextflow.dag.MultipleInputChannelException: Channel fastqs_s has been used twice as an input by process clumpify and another operator at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance( NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.codehaus.groovy.reflection.CachedConstructor. invoke(CachedConstructor.java:83) at org.codehaus.groovy.runtime.callsite.ConstructorSite$ ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105) at org.codehaus.groovy.runtime.callsite.CallSiteArray. defaultCallConstructor(CallSiteArray.java:60) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor( AbstractCallSite.java:235) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor( AbstractCallSite.java:263) at nextflow.dag.DAG.inbound(DAG.groovy:192) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod. invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass. invokeMethod(ClosureMetaClass.java:384) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite. callCurrent(PogoMetaClassSite.java:69) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. callCurrent(AbstractCallSite.java:174) at nextflow.dag.DAG$_addVertex_closure1.doCall(DAG.groovy:131) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.reflection.CachedMethod. invoke(CachedMethod.java:93) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325) at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass. invokeMethod(ClosureMetaClass.java:294) at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022) at groovy.lang.Closure.call(Closure.java:414) at groovy.lang.Closure.call(Closure.java:430) at org.codehaus.groovy.runtime.DefaultGroovyMethods.each( DefaultGroovyMethods.java:2040) at org.codehaus.groovy.runtime.DefaultGroovyMethods.each( DefaultGroovyMethods.java:2025) at org.codehaus.groovy.runtime.DefaultGroovyMethods.each( DefaultGroovyMethods.java:2066) at org.codehaus.groovy.runtime.dgm$162.invoke(Unknown Source) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$ PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite. call(PojoMetaMethodSite.java:56) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. call(AbstractCallSite.java:125) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. callSafe(AbstractCallSite.java:87) at nextflow.dag.DAG.addVertex(DAG.groovy:130) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$ PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent( PogoMetaMethodSite.java:59) at nextflow.dag.DAG.addProcessNode(DAG.groovy:89) at nextflow.dag.NodeMarker.addProcessNode(NodeMarker.groovy:33) at nextflow.dag.NodeMarker$addProcessNode$2.call(Unknown Source) at nextflow.processor.TaskProcessor.createOperator( TaskProcessor.groovy:484) at nextflow.processor.TaskProcessor.run(TaskProcessor.groovy:378) at nextflow.processor.TaskProcessor$run.call(Unknown Source) at nextflow.script.BaseScript.process(BaseScript.groovy:134) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$ PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent( PogoMetaMethodSite.java:59) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent( CallSiteArray.java:52) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. callCurrent(AbstractCallSite.java:154) at org.codehaus.groovy.runtime.callsite.AbstractCallSite. callCurrent(AbstractCallSite.java:174) at _nf_script_cc617d25.run(_nf_script_cc617d25:554) at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:325) at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:156) at nextflow.cli.CmdRun.run(CmdRun.groovy:223) at nextflow.cli.Launcher.run(Launcher.groovy:410) at nextflow.cli.Launcher.main(Launcher.groovy:564) Nov-21 15:31:03.440 [Actor Thread 4] DEBUG nextflow.processor.TaskProcessor
• After stop

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10#issuecomment-346167987, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fwvm1-lwtJDLK7AuQykYhaM52riGvks5s40GIgaJpZM4QlRIu .

-- Carlos Guzman - Graduate Student Bioinformatics & Systems Biology University of California, San Diego

*Phone*: (956) 437-6263 E-mail: cag104@ucsd.edu

[dshechter]

updated the local git clone, still failed but at a different line. I'm heading out shortly but can try more tomorrow. Thanks!

Launching /home/david/cipher-workflow-platform/cipher.nf [naughty_bhabha] - revision: a66dfdd7aa

             RUN INFORMATION

===============================================

mode: rna config: /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/AML2_AML3_DMSO-cipher.cfg fasta: /data/NGS/NGS_Data/Stds/hs/GRch38.fa gtf: /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf library: s read-length: 50

subsampled: true

clumpify: true bbduk: true fastqc: true multiqc: true mapping: true bamcoverage: false downstream_analysis false

threads: 20

aligner: star

ERROR ~ 3

-- Check script 'cipher.nf' at line: 323 or see '.nextflow.log' file for more details [warm up] executor > local ERROR ~ Channel fastqs_s has been used twice as an input by process subsample and another operator

-- Check script 'cipher.nf' at line: 347 or see '.nextflow.log' file for more details

[dshechter]

Tried a test with some inhouse Chipseq data and ended with the same error:

[david@drdisorder fq]$ nextflow run ~/cipher-workflow-platform/cipher.nf --mode chip --config ./cipher.cfg --fa /data/NGS/NGS_Data/Stds/GRCh38.fa --gtf /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf --lib s --readLen 150 --threads 30 --subsample --downstream_analysis false --clumpify false --bamcoverage false N E X T F L O W ~ version 0.25.1 Launching /home/david/cipher-workflow-platform/cipher.nf [zen_spence] - revision: a66dfdd7aa

             RUN INFORMATION

===============================================

mode: chip config: /mnt/raid/NGS/NGS_Data/J153+J634/fq/cipher.cfg fasta: /data/NGS/NGS_Data/Stds/GRCh38.fa gtf: /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf library: s read-length: 150

subsampled: true

clumpify: false bbduk: true fastqc: true multiqc: true mapping: true bamcoverage: false downstream_analysis false

threads: 30

aligner: bbmap

ERROR ~ 0

-- Check script 'cipher.nf' at line: 320 or see '.nextflow.log' file for more details [warm up] executor > local ERROR ~ Channel fastqs_s has been used twice as an input by process subsample and another operator

-- Check script 'cipher.nf' at line: 347 or see '.nextflow.log' file for more details

[c-guzman]

That's really wierd. On my system, everything works out fine and this issue doesn't come up. What version of Nextflow are you using?

On Nov 21, 2017 8:04 PM, "dshechter" notifications@github.com wrote:

Tried a test with some inhouse Chipseq data and ended with the same error:

[david@drdisorder fq]$ nextflow run ~/cipher-workflow-platform/cipher.nf --mode chip --config ./cipher.cfg --fa /data/NGS/NGS_Data/Stds/GRCh38.fa --gtf /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf --lib s --readLen 150 --threads 30 --subsample --downstream_analysis false --clumpify false --bamcoverage false N E X T F L O W ~ version 0.25.1 Launching /home/david/cipher-workflow-platform/cipher.nf [zen_spence] - revision: a66dfdd7aa

         RUN INFORMATION

===============================================

mode: chip config: /mnt/raid/NGS/NGS_Data/J153+J634/fq/cipher.cfg fasta: /data/NGS/NGS_Data/Stds/GRCh38.fa gtf: /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf library: s read-length: 150

subsampled: true

clumpify: false bbduk: true fastqc: true multiqc: true mapping: true bamcoverage: false downstream_analysis false

threads: 30 aligner: bbmap

ERROR ~ 0

-- Check script 'cipher.nf' at line: 320 or see '.nextflow.log' file for more details [warm up] executor > local ERROR ~ Channel fastqs_s has been used twice as an input by process subsample and another operator

-- Check script 'cipher.nf' at line: 347 or see '.nextflow.log' file for more details

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10#issuecomment-346237338, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fwq5xoCV92ZqmFyl222j2-P0WfTjDks5s451dgaJpZM4QlRIu .

[dshechter]

$ conda list |grep nextflow nextflow 0.25.1 0 bioconda

I think that's the latest version in bioconda.

[c-guzman]

I can't reproduce your error with either RNA-seq or ChIP-seq data on my computer or my cluster, and some other people in my lab are running this fine. Can you try running this on a personal computer/laptop using the example data in the cipher example_data folder?

[dshechter]

I downloaded nextflow 0.26.4733 independently of conda. Tried running the script and now have some new errors (below). Maybe you are running an earlier version?

N E X T F L O W ~ version 0.26.1 Launching /home/david/cipher-workflow-platform/cipher.nf [golden_roentgen] - revision: a66dfdd7aa WARN: Process subsample is defined two or more times WARN: Process create_mapping_index is defined two or more times WARN: Process create_mapping_index is defined two or more times WARN: Process create_mapping_index is defined two or more times WARN: Process create_mapping_index is defined two or more times WARN: Process clumpify is defined two or more times WARN: Process bbduk is defined two or more times WARN: Process pre_fastqc is defined two or more times WARN: Process post_fastqc is defined two or more times WARN: Process mapping_bbmap is defined two or more times WARN: Process mapping_bowtie2 is defined two or more times WARN: Process mapping_bwa is defined two or more times WARN: Process mapping_hisat2 is defined two or more times WARN: Process mapping_star is defined two or more times WARN: Process bamCoverage is defined two or more times WARN: Process multiqc is defined two or more times WARN: Process multiqc is defined two or more times WARN: Process multiqc is defined two or more times WARN: Process narrow_peak_calling_WI is defined two or more times WARN: Process narrow_peak_calling_NI is defined two or more times WARN: Process broad_peak_calling is defined two or more times WARN: Process multiqc is defined two or more times

             RUN INFORMATION

===============================================

mode: rna config: /data/NGS/NGS_Data/GEO/GSE85107_Armstrong2016_AML/fastq/AML2_AML3_DMSO-cipher.cfg fasta: /data/NGS/NGS_Data/Stds/hs/GRch38.fa gtf: /data/NGS/NGS_Data/Stds/hs/Homo_sapiens.GRCh38.83.gtf library: s read-length: 50

subsampled: true

clumpify: true bbduk: true fastqc: true multiqc: true mapping: true bamcoverage: true downstream_analysis true

threads: 5

aligner: star

ERROR ~ 3

-- Check script 'cipher.nf' at line: 323 or see '.nextflow.log' file for more details [warm up] executor > local

Workflow completed on: Wed Nov 22 12:45:30 EST 2017 Execution status: Failed Workflow Duration: 1.4s

Submit issues/requests on GitHub or e-mail cag104@ucsd.edu

[dshechter]

I tried it on a small ubuntu VM, but not enough storage space to get everything local.

On my main server, I finally got the example chip data running by eliminating any directory symbolic links in the config file (e.g. went from /data to /mnt/raid/). However that didn't work with the RNA-seq pipeline. I'll keep playing with it! Thanks, must be something along these lines that would make it a machine specific problem.

[dshechter]

Ha - figured it out. You have the RNA-seq script asking for chip-seq annotations in the config file. See lines 323-325 in cipher.nf. For now I added dummy text to the config file, and with explicit path naming it seems to work...

[c-guzman]

WOW. I will fix this asap.

Thanks a ton for catching this!

On Wed, Nov 22, 2017 at 11:20 AM, dshechter notifications@github.com wrote:

Ha - figured it out. You have the RNA-seq script asking for chip-seq annotations in the config file. See lines 323-325 in cipher.nf. For now I added dummy text to the config file, and with explicit path naming it seems to work...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/c-guzman/cipher-workflow-platform/issues/10#issuecomment-346448962, or mute the thread https://github.com/notifications/unsubscribe-auth/AO7fwhSag-E_4heujn9LqOpdPrDzqANfks5s5HQBgaJpZM4QlRIu .

-- Carlos Guzman - Graduate Student Bioinformatics & Systems Biology University of California, San Diego

*Phone*: (956) 437-6263 E-mail: cag104@ucsd.edu

[c-guzman]

Easiest fix was leaving single ended config at 5 columns and pair ended configs at 6 columns. Will update README.