Link checking sometimes fails

[MadVikingGod]

Description

We seem to have too many links to check all at once, and we are only increasing with our changelog. I found this when adding an issue template, with 7 links to github. We are very close to the limits

Environment

• github actions

Steps To Reproduce

1. Add 7 or more links to github in any md or HTML file in the repo
2. Create a PR where Links (Fail Fast) / check-links (pull_request) runs
3. Watch it fail with 429 Network error: Too Many Requests

Expected behavior

This step should not fail.

Possible solutions

1. Add an unscoped github token. This has security implications, but it should be relatively safe without any permissions. This will bump the requests limit to something higher.
2. Split up this test into different sections of the repo. It could be as simple as changelog.md, represents >70 link, vs everything else. This only works if the runners are different instances to get different request limits
3. Ignore some of our links, like https://github.com/open-telemetry/opentelemetry-go/releases/tag/.* . This represents one class of growing links, every release creates a new one, and is also a large number of links we check.

[MrAlias]

+1 for an unscoped min-permissioned token.

[MrAlias]

The link checker is failing due to not having some identity associated with the request: https://github.com/open-telemetry/opentelemetry-go/actions/runs/3969987815/jobs/6805174715

The link in question, https://github.com/open-telemetry/opentelemetry-go/projects, succeeds for me when I make the request as my user. But when I make the request anonymously, I reproduce the error (500) mentioned in the workflow.

I think this both motivates us to start using an access token, but it also points out that by doing so we might miss issues for users not logged into GitHub.

[MrAlias]

The link checker is failing due to not having some identity associated with the request: https://github.com/open-telemetry/opentelemetry-go/actions/runs/3969987815/jobs/6805174715

The link in question, https://github.com/open-telemetry/opentelemetry-go/projects, succeeds for me when I make the request as my user. But when I make the request anonymously, I reproduce the error (500) mentioned in the workflow.

I think this both motivates us to start using an access token, but it also points out that by doing so we might miss issues for users not logged into GitHub.

We should see if the TC can open a support case for this.

[MrAlias]

Interim solution is to ignore this in the link checker.

[MrAlias]

Interim solution is to ignore this in the link checker.

https://github.com/open-telemetry/opentelemetry-go/pull/3620