Restrict merges to PRs older than a certain threshold

[JoshuaKGoldberg]

Vaguely similar to #323, but not quite the same. Could we require PRs be older than some minimum age before merging them?

A few use cases where this'd be very useful:

• Dependabot/Snyk/etc updates, waiting X days in case the community discovers malware in a new package
• Keeping alive for reviews: keeping a PR open for X hours or days so more than one approving reviewer gets a chance to review
• Staging builds: making sure a staging PR and its environment is kept alive for some X hours or days before merge

I'm very motivated to get this in for the first bullet on auto-updates, so if you think this is something a first-time contributor could tackle, I'd love to send a PR! 💖

[chdsbd]

I think this is doable. Maybe not the easiest change for a new contributor because we'd need some cron-like functionality.

I think we'd store a PR for reevaluation after some period (N) and then have a cron job that runs every minute or something and checks for PRs that are old enough to merge.

Here's how we mark a PR for evaluation: https://github.com/chdsbd/kodiak/blob/444145f26106e748f497746a25bfc2314b8d510e/bot/kodiak/queue.py#L558-L560

Do you imagine this would be configurable to only affect PRs opened by specific users? Would repo users still be able to manually merge the PR before the time period is up?

Also, do you have ideas about what the configuration would look like?

[merge.after_period]
# only enforce for specified usernames.
usernames = ["dependabot[bot]", "snyk-bot"]
# only merge PRs after the specified age for days
days = 14

[JoshuaKGoldberg]

Do you imagine this would be configurable to only affect PRs opened by specific users?

Ideally yes.

• Example: it might be necessary to enable this for dependency updates and other bots (and/or user roles?), but not for humans.

Would repo users still be able to manually merge the PR before the time period is up?

Hmm, great question. I would imagine both forms would be desirable. And we'd probably want to be able to configure some list of users (and/or user roles?) that the restriction wouldn't apply to

• Example: dependency updates might be trusted if approved by a non-bot.
• Example: a repository might have a set of admins who are the only ones allowed to merge production hotfixes.

Also, do you have ideas about what the configuration would look like?

TBH I haven't used kodiak yet, just gotten excited about it after seeing it in the Next.js repo 😄. But from looking through your configuration files, a few thoughts...

• days: can that be minutes instead, or failing that, hours?
  • Example: a team might have blue/green deploy builds set up to serve a low percentage from the newer deployment, with a ramp-up occurring within hours if no new crashes/errors are detected
• usernames: would we also want ignored_usernames? If so, how would the two interact?
• Can we also have branches (e.g. branches: ["staging"]) and/or labels (e.g. labels: ["dependencies"])?

[JoshuaKGoldberg]

Closing out my old issues for repos I no longer have context on. If anybody has a need for what this issue was asking about, I'd encourage them/you to file a new issue. Cheers!