search

pulumi / schema-tools

Tools to analyze Pulumi schemas
Apache License 2.0
6 stars 1 forks source link

schema-tools exceed GitHub rate limit #64

Open t0yv0 opened 7 months ago

t0yv0 commented 7 months ago

What happened?

It would be great if schema-tools accepted a local JSON file for comparison instead of polling GItHub API. In the context of bridged provider builds it is possible to fetch this from Git without putting pressure on the GitHub rate limit that we might be facing company-wide. This will make our builds more reliable.

Run EOF=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
ERROR: logging before flag.Parse: E0208 21:43:04.907604   17665 log.go:80] GitHub rate limit exceeded for https://api.github.com/repos/pulumi/pulumi-azuredevops/contents/provider/cmd/pulumi-resource-azuredevops/schema.json?ref=master, try again in 51m5.092418163s.
Error: rate limit exceeded: 403 HTTP error fetching schema from https://api.github.com/repos/pulumi/pulumi-azuredevops/contents/provider/cmd/pulumi-resource-azuredevops/schema.json?ref=master
Usage:
  schema-tools compare [flags]

FAILED https://github.com/pulumi/pulumi-azuredevops/pull/293 FAILED https://github.com/pulumi/pulumi-azuredevops/pull/292 FAILED https://github.com/pulumi/pulumi-databricks/pull/338

Example

See above

Output of pulumi about

N/A

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

iwahbe commented 6 months ago

I believe that each repo has its own rate limit. Both data bricks and azuredevops were hitting rate limits because they were trying to download pulumi-azurerm repeatedly during example generation. Since this was fixed, I don't believe we have seen any rate limit issues from either of them.

t0yv0 commented 6 months ago

That's great news! It certainly still feels more sensible to me for schema-tools to get the content "locally" but clearly in the grand scheme of things this doesn't matter as much.

iwahbe commented 6 months ago

I agree. I actually just merged a change that makes file a valid schema field for URLs. If schema-tools compare took [src1] [src2] (which would make a lot of sense), then this would already work. Instead of adding more flags here, we should consider migrating to that approach.