askimed / nf-test

Simple test framework for Nextflow pipelines
https://www.nf-test.com
MIT License
146 stars 25 forks source link

Order of tests in the snapshot is non-deterministic #275

Open maxulysse opened 3 days ago

maxulysse commented 3 days ago

For example in https://github.com/nf-core/references/pull/39

I have 2 tests in tests/kallisto.nf.test:

Somehow the order of these tests changed in the snapshot file (tests/kallisto.nf.test.snap). (I don't remember if I deleted the snapshots before I regenerated, or used --cleanSnapshot)

And it becomes a bit more complicated to figure out if what I have changed affect one or both tests.

(simple to figure out on simple tests, but could be much more complex and messy on bigger tests).