search

askimed / nf-test

Simple test framework for Nextflow pipelines
https://www.nf-test.com
MIT License
131 stars 17 forks source link

Caching of setup fixtures' results #215

Open Midnighter opened 2 months ago

Midnighter commented 2 months ago

I recently worked on a test that involved using a setup step to untar a rather large database (~8 GB), see here. These tests were causing the GitHub runners to fail because they ran out of space. So that's when I discovered that files for an identical setup step are staged and results generated individually. Leading to more than 24 GB space being used.

In pytest, there is a concept of fixture scope, which can be per function, module, or the entire test session. Results of fixtures are cached and reused accordingly. I think, that'd be a fantastic feature for nf-test as well.

lukfor commented 2 months ago

Thanks, great idea! I will look into this in the next few weeks 👍