viash-io / viash

script + metadata = standalone component
https://viash.io
GNU General Public License v3.0
39 stars 2 forks source link

nxf: multiple output files with same extension overwrite eachother #76

Closed rcannood closed 3 years ago

rcannood commented 3 years ago

Example component src/bar/config.vsh.yaml:

functionality:
  name: bar
  arguments:
    - name: "--input"
      type: file
      example: input.txt
    - name: "--output1"
      type: file
      direction: output
      required: true
      example: output.txt
    - name: "--output2"
      type: file
      direction: output
      required: true
      example: optional.txt
  resources:
    - type: r_script
      dest: script.R
      text: |
        lines <- readr::read_lines(par$input)
        readr::write_lines(lines, par$output1)
        readr::write_lines(lines, par$output2)
platforms:
  - type: docker
    image: rocker/tidyverse:4.0.5
  - type: nextflow

Example workflow main.nf:

nextflow.enable.dsl=2

include  { bar } from "./target/nextflow/bar/main.nf" params(params)

workflow {
    main:
    if (!params.containsKey("input") || params.input == "") {
        exit 1, "ERROR: Please provide a --input parameter."
    }

    output_ = Channel.fromPath(params.input)
        | map { [ "foo", it, params ] }
        | view { [ "DEBUG1", it[0], it[1] ] }
        | bar
        | view { [ "DEBUG2", it[0], it[1] ] }

Running the commands below reveals that the two output files overwrite one another:

$ viash ns build
$ ./run.sh 
N E X T F L O W  ~  version 21.04.1
Launching `main.nf` [nice_torricelli] - revision: 30f7d854fb
executor >  local (1)
[50/2dc682] process > bar:bar_process (foo) [100%] 1 of 1 ✔
[DEBUG1, foo, /home/rcannood/Downloads/example/nextflow.config]
[DEBUG2, foo, [output1:/home/rcannood/Downloads/example/work/50/2dc68212e43b9c9fc050d2ec660ea9/foo.bar.txt]]
[DEBUG2, foo, [output2:/home/rcannood/Downloads/example/work/50/2dc68212e43b9c9fc050d2ec660ea9/foo.bar.txt]]
>> Be careful, multiple outputs from this component!

When there are multiple outputs, the output paths should probably be foo.bar.output1.txt and foo.bar.output2.txt.

example.zip

rcannood commented 3 years ago

Solved by 60e8038