How to use the same samplesheet (containg relativ paths) for both nextflow & nf-test?

[rollf]

Hi, I have a samplesheet that contains the paths to test data relative to the project root directory (i.e. ./tests/fastq.fq). This can be fed into nextflow just fine. I cannot use it for nf-test because lauchnDir is <root>/.nf-test/tests/ec09802a1b5407d9ce413b436ce44403 in there, these paths are not correct. Using stage { symlink <dir-containing-the-fastq-files> } doesn't work neither because the symlink is created in $launchDir/meta (and not in $launchDir directly). Besides, the docs say that file is/will become obsolete.

How can I use the same samplesheet for both nextflow and nf-test?

(I'm sorry if this is explained somewhere, I didn't find any match here or in the docs.)

[lukfor]

Hi! Thank you! This issue should be resolved in version 0.9.0-rc2. We now use <root> as the launchDir, making staging obsolete. I plan to release 0.9.0 tomorrow.

[lukfor]

Btw: you can test it with version 0.9.0-rc2 by installing it using this command:

wget -qO- https://get.nf-test.com | bash -s 0.9.0-rc2

[rollf]

Okay, cool, thank you!

[rollf]

I tried version 2bc23bc73d7f2643bc7a70b042dc52b9f45190ff (9.0.0-rc2) and I'm still not able to run nf-test as expected. Maybe I'm doing it wrong :thinking:. This is how my test data looks like. I'm able to run nextflow ... --input tests/data/tsc/samplesheet.csv but nf-test will still complain.

❯ cat tests/data/tsc/samplesheet.csv
sample,fastq_1,fastq_2
HD200-rep1_S1,./tests/data/tsc/HD200-rep1_S1_R1_001.fastq.gz,./tests/data/tsc/HD200-rep1_S1_R2_001.fastq.gz
HD701-rep1_S2,./tests/data/tsc/HD701-rep1_S2_R1_001.fastq.gz,./tests/data/tsc/HD701-rep1_S2_R2_001.fastq.gz
❯ ls tests/data/tsc/*gz
tests/data/tsc/HD200-rep1_S1_R1_001.fastq.gz  tests/data/tsc/HD200-rep1_S1_R2_001.fastq.gz  tests/data/tsc/HD701-rep1_S2_R1_001.fastq.gz  tests/data/tsc/HD701-rep1_S2_R2_001.fastq.gz
❯ cat nf-test.config
config {
    testsDir "."
    workDir ".nf-test"
    configFile "conf/test.config"
    profile "test,docker"
}
❯ ls tests/stub.main.nf.test
tests/stub.main.nf.test

I get the following errors:

  * -- Entry 1 - fastq_2: the file or directory './tests/data/tsc/HD200-rep1_S1_R2_001.fastq.gz' does not exist.
  * -- Entry 2 - fastq_2: the file or directory './tests/data/tsc/HD701-rep1_S2_R2_001.fastq.gz' does not exist.
  * -- Entry 1 - fastq_1: the file or directory './tests/data/tsc/HD200-rep1_S1_R1_001.fastq.gz' does not exist.
  * -- Entry 2 - fastq_1: the file or directory './tests/data/tsc/HD701-rep1_S2_R1_001.fastq.gz' does not exist.

The launchDir is still <root>.nf-test/tests/4d4e01064dbde7e0ba4382f8f56466c8 (as shown when running nf-test test --verbose --debug)

[rollf]

I resolved this problem by using NXF_FILE_ROOT (docs): The samplesheet should designed to work with nextflow out of the box (i.e. make paths relative to pipeline root dir). But I run nf-test with cd <pipeline>; export NXF_FILE_ROOT=$PWD; nf-test test ....