PR previews: Only build for PRs with specific label(s)

mhilbrunner commented 4 weeks ago

What's the problem this feature will solve?

I'd like to enable PR preview builds for the Godot Documentation hosted on RTD.org. However:

preview builds would be extremely useful, but only on a small subset of PRs (involving tricky markup, tables, images or altering layout/CSS)
we get a large number of contributors, the large majority being one-offs, so trust may be an issue; we don't want to run any potentially malicious code
we get hundreds of PRs and updates to PRs a month, the large majority of which would waste resources needlessly if we were to build all of them

We would like to have a way to run preview builds on a select subset of PRs.

Describe the solution you'd like

If we could designate Github labels to look for to determine whether to run a preview build, that would solve it nicely. Only trusted contributors can edit labels, and we could have "preview-build" or "layout change" labels for this.

If keeping it to a single, non-configurable label like "rtd-preview-build" is easier, that would still be fine. :)

Alternative solutions

An alternative could be to manually request preview builds from a Github bot or in some other way, though this is likely more work to implement.

https://github.com/readthedocs/readthedocs.org/issues/10886 is similar, but different, and may alleviate part of the problem - if I could restrict PR preview builds to contributors with commit rights, I could at least enable the feature. However, a large part of the value would be missing, as we wouldn't get any benefit on the majority of PRs, which are by third parties (which are also often where these reviews would be most helpful).

humitos commented 4 weeks ago

Related, or even maybe duplicate of https://github.com/readthedocs/readthedocs.org/issues/11326

humitos commented 4 weeks ago

In the meantime, you could "Cancel build based on a condition" by following this document: https://docs.readthedocs.io/en/stable/build-customization.html#cancel-build-based-on-a-condition

mhilbrunner commented 4 weeks ago

Related, or even maybe duplicate of #11326

I agree those are very similar, and #11326 seems like it would solve this.

In the meantime, you could "Cancel build based on a condition" by following this document

Thanks, I'll have a look (although it doesn't look like it completely solves the trust issue, as someone's PR could just as well modify those files -- however, it may be a decent stopgap for now, to at least save some resources).

readthedocs / readthedocs.org