search

assemblerflow / flowcraft

FlowCraft: a component-based pipeline composer for omics analysis using Nextflow. :whale::package:
GNU General Public License v3.0
241 stars 44 forks source link

Bash errors when running nextflow with latest version (19.07.0) #217

Open cimendes opened 5 years ago

cimendes commented 5 years ago

All FlowCraft pipelines have been failing in the latest version of nextflow (installed through conda). The errors are fairly general.

For example I created a simple pipeline with just one component with flowcraft build -t "fastqc" -o fastqc and ran it with nextflow run fastqc.nf -profile docker on a single sample. This was my output: 

N E X T F L O W  ~  version 19.07.0
Launching `fastqc.nf` [angry_lovelace] - revision: 1b9c81c955

============================================================
                F L O W C R A F T
============================================================
Built using flowcraft v1.4.2

 Input FastQ                 : 2
 Input samples               : 1
 Reports are found in        : ./reports
 Results are found in        : ./results
 Profile                     : docker

Starting pipeline at Tue Aug 06 15:55:07 UTC 2019

WARN: Process configuration syntax $processName has been deprecated -- Replace `process.$fastqc2_1_1 = <value>` with a process selector
executor >  local (2)
executor >  local (3)
[36/504e43] process > fastqc2_1_1 (SRR7985620)   [ 67%] 2 of 3, failed: 2
executor >  local (4)
[7f/958cca] process > fastqc2_1_1 (SRR7985620)   [ 75%] 3 of 4, failed: 3
[-        ] process > fastqc2_report_1_1         -
executor >  local (4)
[7f/958cca] process > fastqc2_1_1 (SRR7985620)   [ 75%] 3 of 4, failed: 3
[-        ] process > fastqc2_report_1_1         -
[-        ] process > compile_fastqc_status2_1_1 -
executor >  local (5)
[20/fadc40] process > fastqc2_1_1 (SRR7985620)   [ 80%] 4 of 5, failed: 4
[-        ] process > fastqc2_report_1_1         -
[-        ] process > compile_fastqc_status2_1_1 -
executor >  local (6)
[1a/706f14] process > fastqc2_1_1 (SRR7985620)   [ 83%] 5 of 6, failed: 5
[-        ] process > fastqc2_report_1_1         -
[-        ] process > compile_fastqc_status2_1_1 -
[-        ] process > status                     -
executor >  local (7)
[e0/5d4273] process > fastqc2_1_1 (SRR7985620)   [ 86%] 6 of 7, failed: 6
[-        ] process > fastqc2_report_1_1         -
[-        ] process > compile_fastqc_status2_1_1 -
[-        ] process > status                     -
[-        ] process > compile_status_buffer      -
executor >  local (7)
[e0/5d4273] process > fastqc2_1_1 (SRR7985620)   [100%] 7 of 7, failed: 7
[-        ] process > fastqc2_report_1_1         -
[-        ] process > compile_fastqc_status2_1_1 -
[-        ] process > status                     -
[-        ] process > compile_status_buffer      -
[-        ] process > compile_status             -
executor >  local (8)
[a8/143c21] process > fastqc2_1_1 (SRR7985620)   [ 88%] 7 of 8, failed: 7
[-        ] process > fastqc2_report_1_1         -
[-        ] process > compile_fastqc_status2_1_1 -
[-        ] process > status                     -
[-        ] process > compile_status_buffer      -
[-        ] process > compile_status             -
executor >  local (8)
[a8/143c21] process > fastqc2_1_1 (SRR7985620)   [100%] 8 of 8, failed: 8 ✔
[-        ] process > fastqc2_report_1_1         -
[-        ] process > compile_fastqc_status2_1_1 -
[-        ] process > status                     -
[-        ] process > compile_status_buffer      -
[-        ] process > compile_status             -
[-        ] process > report                     -
executor >  local (8)
[a8/143c21] process > fastqc2_1_1 (SRR7985620)   [100%] 8 of 8, failed: 8 ✔
[-        ] process > fastqc2_report_1_1         -
[-        ] process > compile_fastqc_status2_1_1 -
[-        ] process > status                     -
[-        ] process > compile_status_buffer      -
[-        ] process > compile_status             -
[-        ] process > report                     -
[-        ] process > compile_reports            -
Completed at: Tue Aug 06 15:55:35 UTC 2019
Duration    : 27.9s
Success     : true
Exit status : 0
WARN: The channel `create` method is deprecated -- it will be removed in a future release
[ca/91a375] NOTE: Process `fastqc2_1_1 (SRR7985620)` terminated with an error exit status (1) -- Execution is retried (1)
[6d/628071] NOTE: Process `fastqc2_1_1 (SRR7985620)` terminated with an error exit status (1) -- Execution is retried (2)
[36/504e43] NOTE: Process `fastqc2_1_1 (SRR7985620)` terminated with an error exit status (1) -- Execution is retried (3)
[7f/958cca] NOTE: Process `fastqc2_1_1 (SRR7985620)` terminated with an error exit status (1) -- Execution is retried (4)
[20/fadc40] NOTE: Process `fastqc2_1_1 (SRR7985620)` terminated with an error exit status (1) -- Execution is retried (5)
[1a/706f14] NOTE: Process `fastqc2_1_1 (SRR7985620)` terminated with an error exit status (1) -- Execution is retried (6)
[e0/5d4273] NOTE: Process `fastqc2_1_1 (SRR7985620)` terminated with an error exit status (1) -- Execution is retried (7)
[a8/143c21] NOTE: Process `fastqc2_1_1 (SRR7985620)` terminated with an error exit status (1) -- Error is ignored

When looking at the .command.log files of each fastqc process, we obtain the following: 

WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
/home/cimendes/temp/work/a8/143c2139c2925e6e14933c6c3c0fff/.command.run: line 186: uname: command not found
/home/cimendes/temp/work/a8/143c2139c2925e6e14933c6c3c0fff/.command.run: line 138: grep: command not found
/home/cimendes/temp/work/a8/143c2139c2925e6e14933c6c3c0fff/.command.run: line 139: grep: command not found
/home/cimendes/temp/work/a8/143c2139c2925e6e14933c6c3c0fff/.command.run: line 200: date: command not found
Command 'ps' required by nextflow to collect task metrics cannot be found

The bash errors thrown are not very informative and I still haven't been able to figure out what is the underlying cause. This pipeline works well with previous versions of nextflow (tested with 18.10.1).

Input on this issue is very welcomed!

cimendes commented 5 years ago

If i disable the trace file in the nextflow.config, it works (until it dies because trace files are a necessary thing!). Removing either of trace file fields makes no difference. This problem occurs with either container software.

cimendes commented 5 years ago

The source of this issue has been identified and an issue has been opened in nextflow repository. See https://github.com/nextflow-io/nextflow/issues/1289