Workflow proposal for automated runs

[gentlementlegen]

Triggering a run on a organization event implies a few issues when running this plugin:

• it is unreliable, as some repos might not have events (e.g. a small organization) so the plugin never runs and never automatically merges the PR
• it is resource heavy, when a lot of events are triggered (e.g. big organization) so the plugin runs a lot, consuming lot of compute time for nothing
• it requires more than one issue, since actions on the same issue would simply refresh the delay of the activity not considered as stale

Proposal

We already had a long talk about not having cron tasks. However I think it is suitable for this plugin, because:

• it scales well, regardless of the activity of the organization (e.g. only one run a day)
• it is assured to run, regardless of the amount of issues or no activity
• it will run only when necessary (perfect control over resources used)

To achieve so, my proposal would be to have a separate workflow which only purpose is to trigger a run once a day. If no pull-request is found, the workflow would get disabled thus not run anymore. On the event pull_request.opened or pull_request.reopened, the workflow would be enabled again. One run every 24 hours seems reasonable and realistic for our needs.

[0x4007]

No for the same reasons we discussed. To repeat my main concern, imagine a future where we have 1000 partners. maybe 5 will be power users, maybe 100 will be "average" users, and the other 895 will be dead orgs. cron is horrible for scaling for this reason. You're wasting a ton of resources on the dead orgs.

[gentlementlegen]

On the dead orgs, the CRON would never run, so I am not sure where resources are being wasted? Plus given the use case the company with power users will maybe trigger 5000 runs a day, which probably will hit GitHub API limits and saturate the Action runs. Plus if they are not running with open source, they use all their Action quota quickly.

[0x4007]

On the dead orgs, the CRON would never run, so I am not sure where resources are being wasted? Plus given the use case the company with power users will maybe trigger 5000 runs a day, which probably will hit GitHub API limits and saturate the Action runs. Plus if they are not running with open source, they use all their Action quota quickly.

The limit is 5000 per hour. We can make optimizations if some day in the far future it becomes a tangible problem.

[gentlementlegen]

Sure. Also remember that each run is one KV use as well, which is 1000 a day if I recall.

[0x4007]

On the dead orgs, the CRON would never run

This also does not make sense to me. The purpose of the CRON is to check if it should react, and then to react. Events based does not need to check, it just reacts.

With CRON you need to check every org for if the job needs to react.

[gentlementlegen]

The purpose of the CRON is to trigger a run, which if it realizes that there is nothing to do, will disable the CRON.

Every time the workflow is triggered all the orgs and repos and checked either way, not sure what you mean by this.

[0x4007]

Every time the workflow is triggered all the orgs and repos and checked either way

This is wrong, it should only check the organization which invoked the event.

[gentlementlegen]

It reads the configuration where you can specify what organization / repos you want to be watched. Otherwise we cannot support cross organization or non-organization repositories.

[0x4007]

Otherwise we cannot support cross organization or non-organization repositories.

I don't understand this but the only time/place that an update should be made by the bot is where activity is occurring from human contributors.

[gentlementlegen]

What I mention is in this pull request: https://github.com/ubiquibot/automated-merging/pull/4

[0x4007]

Otherwise we cannot support cross organization or non-organization repositories.

Let's not do this. I think it simplifies implementation a lot if we only manage the active organization with the event source. I think its very unusual that a single company has multiple GitHub organizations. We do because we're developing a product intended to be used on GitHub and it was originally just for GitHub QA testing.

[ubiquibot[bot]]

# Issue was not closed as completed. Skipping.

[ubiquibot-dev[bot]]

# Issue was not closed as completed. Skipping.

[0x4007]

+ Evaluating results. Please wait...

@gentlementlegen I noticed that it posted a message "evaluating results" but it shouldn't do that work at all when closed as not planned. We should also change the message to something like:

# No rewards because issue was closed as not planned.

[gentlementlegen]

Okay I can make a ticket on that regard, will take care of it.