web-platform-tests / wpt.live

A live version of the web-platform-tests project
https://wpt.live/
15 stars 11 forks source link

pr-preview sychronize fails to handle pull requests outside its window #36

Closed stephenmcgruer closed 4 years ago

stephenmcgruer commented 4 years ago

The pr-preview script in WPT is responsible for determining what pull requests should be mirrored. It runs every 5 minutes on a cronjob, and looks at an 8 minute window of changed pull requests to decide what to do.

The problem with this is that if pr-preview ever breaks or doesn't run in some 8 minute window, for whatever reason, then PRs are missed. This can miss both PRs that are closed (so should have their deployment deleted) as well as PRs that are opened (and just won't get a deployment). The former is a larger problem than the latter.

Right now, there are 103 PRs with a refs/prs-open/{pr} ref for them. Of those, only 59 are open and 44 are closed. At ~400MB a sync, that's 18GB of unnecessary disk space! (see https://github.com/web-platform-tests/wpt.live/issues/35).

It's not feasible to scan all PRs, but what we can do is scan the existing refs (along with the window) and check if any should be updated or deleted. This will definitely catch deletions, but could still miss additions - however those can be trivially fixed by anyone commenting on the pull request or close/re-opening it, so it seems more reasonable to miss them.