operator-framework / operator-controller

A new and improved management framework for extending Kubernetes with Operators
https://operator-framework.github.io/operator-controller/
Apache License 2.0
61 stars 53 forks source link

:sparkles: feat: implement initial benchmark tests #1071

Open OchiengEd opened 2 months ago

OchiengEd commented 2 months ago

Description

This PR is for implementing benchmarking tests which will be used in CI to evaluate change in performance / compute resource utilization as new features are implemented.

Reviewer Checklist

Relates to #920

netlify[bot] commented 2 months ago

Deploy Preview for olmv1 ready!

Name Link
Latest commit 1075eb334f8efe39ccd0acd826a06d534e4b7f36
Latest deploy log https://app.netlify.com/sites/olmv1/deploys/66b52ff7e13e21000831fa97
Deploy Preview https://deploy-preview-1071--olmv1.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Project coverage is 75.23%. Comparing base (6864b0c) to head (1075eb3). Report is 22 commits behind head on main.

Files Patch % Lines
internal/rukpak/source/image_registry.go 0.00% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1071 +/- ## ========================================== - Coverage 75.28% 75.23% -0.06% ========================================== Files 35 35 Lines 1914 1914 ========================================== - Hits 1441 1440 -1 - Misses 330 331 +1 Partials 143 143 ``` | [Flag](https://app.codecov.io/gh/operator-framework/operator-controller/pull/1071/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=operator-framework) | Coverage Ξ” | | |---|---|---| | [e2e](https://app.codecov.io/gh/operator-framework/operator-controller/pull/1071/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=operator-framework) | `57.41% <0.00%> (-0.06%)` | :arrow_down: | | [unit](https://app.codecov.io/gh/operator-framework/operator-controller/pull/1071/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=operator-framework) | `50.78% <0.00%> (ΓΈ)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=operator-framework#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

OchiengEd commented 2 months ago

In my observation, these seem to be the largest contributors to the duration of a Reconcile

Totally agree. I was intent on going through the code base and identifying where we have more CPU / memory intensive tasks. However, to start working on GH actions, I wanted to have something in place.

OchiengEd commented 2 months ago

While reviewing, consider:

  1. What should be threshold for an increase in computational resource utilization?
  2. And should we error when above threshold is exceeded?
  3. Should we add a comment with the benchmark results?
  4. Informational logs in the function being benchmarked will break the benchmark format. As a workaround, should we remove the comments altogether or suppress them when benchmarks are running?
OchiengEd commented 2 months ago

During a meeting on the benchmarking of operator-controller functions held on 08/13/2024, the following action items were identified:

openshift-merge-robot commented 1 month ago

PR needs rebase.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.