search

sourmash-bio / sourmash_plugin_branchwater

fast, multithreaded sourmash operations: search, compare, and gather.
GNU Affero General Public License v3.0
15 stars 2 forks source link

set up criterion/asv benchmarks? #215

Open bluegenes opened 7 months ago

bluegenes commented 7 months ago

..to facilitate comparisons between any utility function changes + prevent regression when developing.

would be good to set up some macro benchmarks too, e.g. using SRR606249.sig.gz in test-data as query...

bluegenes commented 7 months ago

Note, can't use criterion with cdylib-only build. Recommendations online to add rlib, but that doesn't build on my system (arm mac)

ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

could probably get around this - not sure if worthwhile

ctb commented 7 months ago

see also: https://github.com/sourmash-bio/sourmash_plugin_branchwater/issues/214#issuecomment-1950777360 which says -

scripts in /home/ctbrown/scratch3/SRR606249-feb13-2024

github repo for scripts: https://github.com/sourmash-bio/2024-benchmark-branchwater-plugin

srun command I used:

srun -p bmh --time=24:00:00 --nodes=1 --cpus-per-task=64 --mem=50GB --pty /bin/bash

I feel like we could probably do them manually before each release easily enough. The 45 minute one (sourmash gather) should not be automated :)