search

askimed / nf-test

Simple test framework for Nextflow pipelines
https://www.nf-test.com
MIT License
134 stars 19 forks source link

Conda environment are re-created for each test #231

Open maxulysse opened 1 month ago

maxulysse commented 1 month ago

Running the tests for untar in the nf-core/modules repo, with the following command:

nf-test test --verbose --debug --profile "+conda" modules/nf-core/untar

Four tests are launched: test_untar, test_untar_onlyfiles, test_untar - stub and test_untar_onlyfiles - stub.

And I see four similar messages on my terminal with 4 different conda cache location.

Creating env using conda: /home/max/workspace/modules/modules/nf-core/untar/tests/../environment.yml [cache /home/max/tests/5914f5c10741c2a84844fa7727ecaf5c/work/conda/untar-6ec82032d38a28f610b86307ce4a0ff3]

Any recommendation on how to deal with this properly, and avoid re-creating conda env multiple time?

lukfor commented 1 month ago

At the moment, each test uses its own work directory.

We could consider setting a work directory alias for each test suite or test case. This way, nf-test would create a work directory under .nf-test/share/<workdir alias>, allowing tests with the same work directory alias to reuse the directory.

What do you think?

maxulysse commented 1 month ago

That's already a good progress, but I'm guessing in such a case, if I have already the conda env it will be recomputed anyway, right? Maybe it's something that we can deal with using cachedir within https://www.nextflow.io/docs/latest/config.html#scope-conda