dhiller commented 9 months ago

botreview aims to automate reviews that follow simple patterns.

Motivation of this is to check automatically created PRs like image bumps, prow bumps, kubevirtci bumps etc., i.e. all PRs that follow simple patterns, are tedious to review but still required to check.

If anything beyond those patterns appears, it will backoff and add a comment with suspicious changes it found.

Note: botreview only acts as a first gate to filter out those PRs that really require a humans review!

Roadmap

M1
- [x] get https://github.com/kubevirt/project-infra/pull/2448 merged
- [x] testing 1st round
  - [x] see how it goes, monitor the logs, monitor stability
  - outcome: seems to behave well, no interference so far to be noticeable
M2
- [x] secret
  - [x] add to kubevirt/secrets
  - [x] integrate into kubevirt/project-infra
- [x] testing 2nd round
  - [x] sporadically set --dry-run to false and see how it goes, monitor the logs
  - [x] outcome: refine image_bump.go to further look out for image spec changes: https://github.com/kubevirt/project-infra/pull/3118
M3
- [ ] set --dry-run=false in project-infra deployment see how it goes, monitor the logs
M4
- [ ] plugins config: extend from project-infra to other projects
  - [ ] kubevirt/kubevirtci
  - [ ] kubevirt/kubevirt
M5
- [ ] create kubevirt-review-bot GitHub service account
- [ ] replace @dhiller token secret with a proper bot account token [^1]

How to run

It can be run in two ways:

as a command

Example run:

go run ./robots/cmd/botreview/... --dry-run=false --github-token-path ... --pr-number=2485 --repo=project-infra

If --dry-run=false is set, it will attach a comment to the review that states the review result.

Result: https://github.com/kubevirt/project-infra/pull/2485#issuecomment-1327188198

Note: using --dry-run=true or omitting the flag will run the tool in dry-run which will just print out the result)

as an external plugin for prow

It will react on pull request events for the repos it's configured for in the plugins config and will review simple PRs automatically.

Note that the deployment currently has `--dry-run=true enabled, since we want to test it's stability gradually.

Review strategies

Review strategies implemented today:

kubevirt/project-infra
- [x] prow job image bumps
- [x] prow bumps
- [ ] OWNERS file updates in prowjob configs
kubevirt/kubevirt
- [x] kubevirt kubevirtci bumps
  - [ ] KubeVirtCI Bump: deleted files inside the cluster-up folder are ok
- [ ] make rpm-deps
- [ ] RPM mirror PRs https://github.com/kubevirt/project-infra/pull/3131
kubevirt/kubevirtci
- [ ] cdi bump
- [ ] patch version bump

Note: It can also handle mixed PRs, where each change abides by the rules of the strategy.

[^1]: required since kubevirt-bot can't lgtm their own PRs

dhiller commented 9 months ago

/area automation /area prow

dhiller commented 9 months ago

/kind tracker

dhiller commented 9 months ago

From the first round of testing I got these results:

so far plugin seems to behave well, at current it had memory consumption of about 65 Mib at total over one night, seemingly caused by the checkout of the repos
image_bump.go seems to be a bit unselective. It also took into account normal prowjob change PRs that were unrelated to image bump. We should probably be more specific when selecting relevant changes, i.e. look for changes to image: config specifically.

dhiller commented 9 months ago

Need to improve kubevirtci bump selection: https://github.com/kubevirt/kubevirt/pull/10747#issuecomment-1851616047

kubevirt-bot commented 5 months ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

kubevirt-bot commented 4 months ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

kubevirt-bot commented 3 months ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

/close

kubevirt-bot commented 3 months ago

@kubevirt-bot: Closing this issue.

In response to [this](https://github.com/kubevirt/project-infra/issues/3100#issuecomment-2154717871): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

dhiller commented 3 months ago

/reopen

We still want to do this.

kubevirt-bot commented 3 months ago

@dhiller: Reopened this issue.

In response to [this](https://github.com/kubevirt/project-infra/issues/3100#issuecomment-2154731354): >/reopen > >We still want to do this. Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

dhiller commented 3 months ago

/remove-lifecycle rotten

kubevirt-bot commented 1 week ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

kubevirt / project-infra

botreview - automate simple reviews #3100

Roadmap

How to run

as a command

as an external plugin for prow

Review strategies