Avoid redundant CI work by only building main and pull requests

[PeterJCLaw]

There's very little benefit to building every pushed commit and this will mostly duplicate CI runs for open pull requests.