search

AfshinLab / BLR

MIT License
5 stars 0 forks source link

Speed up tests #12

Closed marcelm closed 4 years ago

marcelm commented 4 years ago

This speeds up running the tests from 210 seconds to 66 seconds on my machine.

This is achieved by introducing a "workdir" fixture that runs the pipeline up to the ready-for-variant-calling BAM file (mapped.calling.bam). The fixture is scoped at module level, that is, it runs only once, allowing us to re-use the already computed parts of the pipeline in some of the slowest tests.

A second change is that the test data is now unpacked into blr-testdata instead of testdata. This is for consistency and to prepare adding a Snakefile under testdata/Snakefile. You may need to adjust your symlinks if you have any, like I do.

Before (total 215s):

29.24s call     tests/test_cli.py::test_BQSR
26.27s call     tests/test_cli.py::test_call_variants[gatk]
25.16s call     tests/test_cli.py::test_haplotag[blr]
21.31s call     tests/test_cli.py::test_call_variants[bcftools]
20.97s call     tests/test_cli.py::test_haplotag[whatshap]
20.10s call     tests/test_cli.py::test_call_variants[freebayes]
17.73s call     tests/test_cli.py::test_final_compressed_reads_exist
17.36s call     tests/test_cli.py::test_plot_figures
10.16s call     tests/test_cli.py::test_mappers[bowtie2]
6.79s call     tests/test_cli.py::test_mappers[ema]
6.13s call     tests/test_cli.py::test_mappers[bwa]
5.79s call     tests/test_cli.py::test_mappers[minimap2]
3.65s call     tests/test_cli.py::test_trim_blr
2.28s call     tests/test_cli.py::test_trim_stlfr
1.05s call     tests/test_cli.py::test_trim_tenx
...

After (total 67s):

16.05s setup    tests/test_cli.py::test_default_read_mapper
8.05s call     tests/test_cli.py::test_BQSR
7.40s call     tests/test_cli.py::test_call_variants[gatk]
5.28s call     tests/test_cli.py::test_nondefault_read_mappers[ema]
4.71s call     tests/test_cli.py::test_nondefault_read_mappers[bwa]
4.45s call     tests/test_cli.py::test_call_variants[bcftools]
4.38s call     tests/test_cli.py::test_nondefault_read_mappers[minimap2]
4.07s call     tests/test_cli.py::test_haplotag[blr]
3.68s call     tests/test_cli.py::test_haplotag[whatshap]
2.36s call     tests/test_cli.py::test_plot_figures
2.28s call     tests/test_cli.py::test_trim_stlfr
1.62s call     tests/test_cli.py::test_call_variants[freebayes]
1.02s call     tests/test_cli.py::test_trim_tenx
...

Note that the runtime of 16 seconds for test_default_read_mapper is for the setup. That is the first time the workdir fixture is used in the module, so this runs the workdir fixture for the first time.

pontushojer commented 4 years ago

It all look good to me 👍 I will go ahead and merge