cloudflare / pint

Prometheus rule linter/validator
https://cloudflare.github.io/pint/
Apache License 2.0
826 stars 47 forks source link

`pint ci` spams GitHub's API which replies with status code 422 #699

Open pauloborges opened 11 months ago

pauloborges commented 11 months ago

Hello! It seems pint ci is spamming GitHub's API to post inline comments about the rules.

I have a set of 60+ rules, and Pint manages to post 50-ish comments before being rate-limited. Here's an excerpt of the output:

level=info msg="Pull request review created" status="200 OK"
level=info msg="Creating review comments"
level=fatal msg="Execution completed with error(s)" error="submitting reports: POST https://api.github.com/repos/xxx/xxx/pulls/xxx/comments: 422 Validation Failed [{Resource:PullRequestReviewComment Field:base Code:abuse Message:}]"

And here is the documentation that tells GitHub may return 422 on this endpoint if the caller is spamming it: https://docs.github.com/en/rest/pulls/comments?apiVersion=2022-11-28#create-a-review-comment-for-a-pull-request--status-codes

prymitive commented 1 month ago

There's a maxComment option to limit the number of comments - https://cloudflare.github.io/pint/configuration.html#github-options but 422s could be handled explicitly here too.