sourmash-bio / sourmash

Quickly search, compare, and analyze genomic and metagenomic data sets.
http://sourmash.readthedocs.io/en/latest/
Other
476 stars 79 forks source link

CI: enable codspeed for Python #3262

Open luizirber opened 4 months ago

luizirber commented 4 months ago

Follow-up on #3231 with Python perf tracking.

This uses pytest-benchmark instead of asv, but I'm still not sure that's a good idea, because pytest-benchmark doesn't track memory usage as asv does.

codecov[bot] commented 4 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 86.74%. Comparing base (f621726) to head (19a0ccd).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## latest #3262 +/- ## ======================================= Coverage 86.73% 86.74% ======================================= Files 136 136 Lines 15871 15871 Branches 2728 2728 ======================================= + Hits 13766 13767 +1 + Misses 1796 1795 -1 Partials 309 309 ``` | [Flag](https://app.codecov.io/gh/sourmash-bio/sourmash/pull/3262/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sourmash-bio) | Coverage Δ | | |---|---|---| | [hypothesis-py](https://app.codecov.io/gh/sourmash-bio/sourmash/pull/3262/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sourmash-bio) | `25.40% <ø> (ø)` | | | [python](https://app.codecov.io/gh/sourmash-bio/sourmash/pull/3262/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sourmash-bio) | `92.37% <ø> (ø)` | | | [rust](https://app.codecov.io/gh/sourmash-bio/sourmash/pull/3262/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sourmash-bio) | `62.18% <ø> (+0.03%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sourmash-bio#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codspeed-hq[bot] commented 4 months ago

CodSpeed Performance Report

Merging #3262 will create unknown performance changes

Comparing lirber/codspeed_python (19a0ccd) with latest (f621726)

Summary

⁉️ 21 dropped benchmarks

:warning: _Please fix the performance issues or acknowledge them on CodSpeed._

Benchmarks breakdown

Benchmark latest lirber/codspeed_python Change
⁉️ force with valid kmers 13.9 ms N/A N/A
⁉️ invalid kmers 10.3 ms N/A N/A
⁉️ lowercase 13.9 ms N/A N/A
⁉️ valid 13.9 ms N/A N/A
⁉️ force with valid kmers 396.4 ms N/A N/A
⁉️ invalid kmers 396.8 ms N/A N/A
⁉️ lowercase 396.3 ms N/A N/A
⁉️ valid 396.4 ms N/A N/A
⁉️ abund0_ani_ci0 560.5 µs N/A N/A
⁉️ abund0_ani_ci1 581.5 µs N/A N/A
⁉️ abund1_ani_ci0 770.9 µs N/A N/A
⁉️ abund1_ani_ci1 793.5 µs N/A N/A
⁉️ intersection 18.2 µs N/A N/A
⁉️ intersection_size 15.4 µs N/A N/A
⁉️ large intersection 9.1 ms N/A N/A
⁉️ large intersection btree 18 ms N/A N/A
⁉️ large intersection_size 7.5 ms N/A N/A
⁉️ large intersection_size btree 17.8 ms N/A N/A
⁉️ load nodegraph 187 µs N/A N/A
⁉️ save compressed nodegraph 1.4 ms N/A N/A
... ... ... ... ...


:information_source: _Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks._