nextflow-io / nextflow

A DSL for data-driven computational pipelines
http://nextflow.io
Apache License 2.0
2.69k stars 621 forks source link

NEXTFLOW SPLITTING COMMAND INTO MULTIPLE LINES IN .command.sh #3169

Closed MalcolmorianVII closed 2 years ago

MalcolmorianVII commented 2 years ago

I am facing an issue with nextflow

Problem: I have nextflow version 22.04.5.5708 and I am running a nextflow script using the following command: nextflow run core_bioinfo.nf Script:

nextflow.enable.dsl=2

workflow {
   ch_infiles = Channel.fromPath(params.infiles,checkIfExists:true)
   ch_api = Channel.fromPath(params.api,checkIfExists:true)

   today()
   mk_today(ch_infiles,today.out) | view
   query_api(ch_api,today.out,ch_infiles)
}

process today {
   tag "Get todays folder"

   output:
   stdout emit: day

   script:
   """
   echo \$(date +"%Y.%m.%d")
   """
}

process mk_today {
   tag "Make todays folder"

   input:
   path seq_dir
   val day
   output:
   stdout

   script:
   """
   mkdir -p ${seq_dir}/${day}
   """
}

process query_api {

   input:
   path api
   val day
   val seq_dir

   output:
   file("${day}.sample_source_sample_pcrs.csv")

   script:
   """
   python ${api} > ${seq_dir}/${day}/${day}.sample_source_sample_pcrs.csv
   """
}

Running this script is giving me the following output:

N E X T F L O W  ~  version 22.05.0-edge
Launching `core_bioinfo.nf` [angry_montalcini] DSL2 - revision: a74f4aaa3a
executor >  local (3)
[d3/8df354] process > today (Get todays folder)            [100%] 1 of 1 ✔
[60/029801] process > mk_today (Make todays folder)        [100%] 1 of 1 ✔
[d9/846551] process > query_api (Get covid cases from API) [  0%] 0 of 1

Error executing process > 'query_api (Get covid cases from API)'

Caused by:
  Process `query_api (Get covid cases from API)` terminated with an error exit status (1)

Command executed:

executor >  local (3)
[d3/8df354] process > today (Get todays folder)            [100%] 1 of 1 ✔
[60/029801] process > mk_today (Make todays folder)        [100%] 1 of 1 ✔
[d9/846551] process > query_api (Get covid cases from API) [100%] 1 of 1, failed: 1 ✘

Error executing process > 'query_api (Get covid cases from API)'

Caused by:
  Process `query_api (Get covid cases from API)` terminated with an error exit status (1)

Command executed:

  python get_covid_cases_from_api.py > /home/bkutambe/data/seqbox/infiles/2022.09.01
  /2022.09.01
  .sample_source_sample_pcrs.csv

Command exit status:  1

Command output:
  (empty)

Command error:
  .command.sh: line 2: /home/bkutambe/data/seqbox/infiles/2022.09.01: Is a directory

Work dir:
  /home/bkutambe/Documents/Core Bioinfo/work/d9/846551a8367ec0c0ce724e83d91edb

What I have done: Checking /home/bkutambe/Documents/Core Bioinfo/work/d9/846551a8367ec0c0ce724e83d91edb/.command.sh reveals this:

#!/bin/bash -ue
python get_covid_cases_from_api.py > /home/bkutambe/data/seqbox/infiles/2022.09.01
/2022.09.01
.sample_source_sample_pcrs.csv

My observation: the problem is nextflow is splitting the command of process query_api into 3 separate lines in the .command.sh.And running as it is,is what is giving that error. How can this be solved?

MalcolmorianVII commented 2 years ago

I got help on the issue.Turns out echo \$(date +"%Y.%m.%d") is appending a "\n".Fixed it by using echo -n to suppress the newline character