ordo-one / package-benchmark

Swift benchmark runner with many performance metrics and great CI support
Apache License 2.0
326 stars 25 forks source link

feat: Add more thorough setup/teardown hooks #153

Closed hassila closed 1 year ago

hassila commented 1 year ago

Description

Adding support for global, shared and local setup/teardown hooks in a systemic manner.

Fixes https://github.com/ordo-one/package-benchmark/issues/142

How Has This Been Tested?

Manual testing, added sample.

Minimal checklist:

github-actions[bot] commented 1 year ago

Pull request benchmark comparison [ubuntu-latest] with 'main' run at 2023-04-21T08:57:32+00:00 Pull request had performance regressions

codecov[bot] commented 1 year ago

Codecov Report

Merging #153 (ed8d777) into main (c838d01) will decrease coverage by 6.98%. The diff coverage is 70.21%.

:exclamation: Current head ed8d777 differs from pull request most recent head 17367c9. Consider uploading reports for the commit 17367c9 to get more accurate results

Additional details and impacted files [![Impacted file tree graph](https://codecov.io/gh/ordo-one/package-benchmark/pull/153/graphs/tree.svg?width=650&height=150&src=pr&token=hXHmhEG1iF&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one)](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one) ```diff @@ Coverage Diff @@ ## main #153 +/- ## ========================================== - Coverage 78.66% 71.69% -6.98% ========================================== Files 25 26 +1 Lines 2695 3433 +738 ========================================== + Hits 2120 2461 +341 - Misses 575 972 +397 ``` | [Impacted Files](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one) | Coverage Δ | | |---|---|---| | [Benchmarks/Histogram/Histogram.swift](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one#diff-QmVuY2htYXJrcy9IaXN0b2dyYW0vSGlzdG9ncmFtLnN3aWZ0) | `44.96% <ø> (ø)` | | | [Sources/Benchmark/Benchmark.swift](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one#diff-U291cmNlcy9CZW5jaG1hcmsvQmVuY2htYXJrLnN3aWZ0) | `72.34% <60.00%> (+1.23%)` | :arrow_up: | | [Sources/Benchmark/BenchmarkRunner.swift](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one#diff-U291cmNlcy9CZW5jaG1hcmsvQmVuY2htYXJrUnVubmVyLnN3aWZ0) | `64.41% <75.00%> (+1.51%)` | :arrow_up: | | [Impacted Files](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one) | Coverage Δ | | |---|---|---| | [Benchmarks/Histogram/Histogram.swift](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one#diff-QmVuY2htYXJrcy9IaXN0b2dyYW0vSGlzdG9ncmFtLnN3aWZ0) | `44.96% <ø> (ø)` | | | [Sources/Benchmark/Benchmark.swift](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one#diff-U291cmNlcy9CZW5jaG1hcmsvQmVuY2htYXJrLnN3aWZ0) | `72.34% <60.00%> (+1.23%)` | :arrow_up: | | [Sources/Benchmark/BenchmarkRunner.swift](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one#diff-U291cmNlcy9CZW5jaG1hcmsvQmVuY2htYXJrUnVubmVyLnN3aWZ0) | `64.41% <75.00%> (+1.51%)` | :arrow_up: | ------ [Continue to review full report in Codecov by Sentry](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one). Last update [c838d01...17367c9](https://codecov.io/gh/ordo-one/package-benchmark/pull/153?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ordo-one).