[BUG] Output path is out of the scope of process working directory

[emirerkol]

Describe the bug A clear and concise description of what the bug is.

When given a path for the loom files to be published (defined in .config file), an error occurs at the SCENIC:MERGE_MOTIF_TRACK_LOOMS step if an absolute path outside the Nextflow working directory is defined. This can be easily avoided if only a filename is given so the file can be published inside the Nextflow working directory.

To Reproduce Steps to reproduce the behavior:

1. Configure with these options:

Make a Nextflow .config file for scenic.

nextflow config \
   -profile hg38,scenic,scenic_use_cistarget_motifs,scenic_use_cistarget_tracks,qsub,singularity vib-singlecell-nf/vsn-pipelines \
   > nextflow.config

Define paths for scenicOutputLoom, scenicScopeOutputLoom (only important part in the config file is given).

scenic {
         scenicOutputLoom = '/user/leuven/330/vsc33048/eerkol/GRN/redone/SCENIC_output.loom'
         scenicScopeOutputLoom = '/user/leuven/330/vsc33048/eerkol/GRN/redone/SCENIC_SCope_output.loom'
            }

2. Run using this entry point:

cd /user/leuven/330/vsc33048/eerkol/GRN/redone

nextflow -C nextflow.config run \
   vib-singlecell-nf/vsn-pipelines \
      -entry scenic

3. See error:

Error executing process > 'scenic:SCENIC:MERGE_MOTIF_TRACK_LOOMS (1)'

Caused by:
  File `/user/leuven/330/vsc33048/eerkol/GRN/redone/SCENIC_output.loom` is out of the scope of process working dir: /ddn1/vol1/staging/leuven/stg_00071/eerkol/GRN/redone/work/83/d2d5dbc58f2fba7c05e44293d374c5

Command executed:

  /user/leuven/330/vsc33048/.nextflow/assets/vib-singlecell-nf/vsn-pipelines/src/scenic/bin/merge_motif_track_loom.py             --loom_motif multi_runs_regulons_auc_mtf.loom             --loom_track multi_runs_regulons_auc_trk.loom             --loom_output /user/leuven/330/vsc33048/eerkol/GRN/redone/SCENIC_output.loom

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /ddn1/vol1/staging/leuven/stg_00071/eerkol/GRN/redone/work/83/d2d5dbc58f2fba7c05e44293d374c5

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

Expected behavior The error comes up pretty late in the process thus the defined path may be checked after when the Nextflow working directory is first created for the potential error. Maybe a default file name can be assigned instead of the given output in the .config file.

Please complete the following information:

• OS: CentOS Linux 7 (Core)
• Nextflow Version: 20.04.1
• vsn-pipelines Version: 0.22.0

Additional context This issue can be avoided if just a filename is given for the output files like given in the config file as default.

[cflerin]

Hey @emirerkol , thanks for reporting this. Indeed, you already figured out that you cannot put a full path into this parameter. My first thought would be to say "you're using it wrong" by doing this, but I guess that it is not really clear, so probably not fair to blame you entirely. There should have been a default value in there already when you generated the config and I usually don't change it. I wonder if maybe we should remove these parameters altogether and let the pipeline determine the output file names (which is what happens in the other workflows).

[emirerkol]

Hi @cflerin, indeed maybe the best option is to remove the parameters (or maybe a warning to not give a full path).