nodejs / community-committee

The Node.js Community Committee (aka CommComm)
MIT License
259 stars 70 forks source link

test: add nightly markdown checking #640

Closed bnb closed 1 year ago

bnb commented 3 years ago

Adds nightly markdown link checking. Helps prevent broken links that are left alone forever.

bnb commented 3 years ago

currently failing because we're getting throttled by GitHub since we have so many GitHub links in this repository 😅

bnb commented 3 years ago

This is currently completely blocked unless I can either ignore a path (/meetings) or batch requests. I've made issues (https://github.com/tcort/markdown-link-check/issues/102 and https://github.com/tcort/markdown-link-check/issues/101, respectively) for these in the correct repository. Until they're addressed, I don't think we can merge this. I'm totally open to alternative Actions that allow us to do this, this was just the best one I've found.

mmarchini commented 3 years ago

I think we can do both using the config-file option, it takes a JSON file to configure the underlying markdown-link-check npm package: https://github.com/tcort/markdown-link-check#config-file-format.

{
    "ignorePatterns": [
        {
            "pattern": "^http://example.net"
        }
    ],
    "replacementPatterns": [
        {
            "pattern": "^.attachments",
            "replacement": "file://some/conventional/folder/.attachments"
        }
    ],
    "httpHeaders": [
        {
            "urls": [
                "https://example.com"
            ],
            "headers": {
                "Authorization": "Basic Zm9vOmJhcg==",
                "Foo": "Bar"
            }
        }
    ]
}

The file probably would need to be generated on each run. Something like (not tested, this could be wildly wrong):

  - run: |
    cat <<EOF > config.json
    {
      "ignorePatterns": [
      {
        "pattern": "^http://github.com/${{ github.repository }}/tree/.*/meetings/.*"
      }
      ],
      "httpHeaders": [
        {
          "urls": [
            "https://github.com"
          ],
          "headers": {
            "Authorization": "Bearer ${{ secrets.GITHUB_TOKEN }}",
          }
        }
      ]
    }
    EOF
bnb commented 3 years ago

@mmarchini could you expand on why it'd need to be generated on each run? The HTTP Header part may be enough itself, I wasn't able to find any docs saying that was enabled/allowed on GitHub's side 🤔

mmarchini commented 3 years ago

Because we receive a new token on each run, and since the file is a JSON we can't set the authentication headers programatically on the config file. If markdown-link-check accepted JS files as well for config (like webpack and eslint do), we wouldn't need to generate on each run.

bnb commented 3 years ago

ah, that makes sense 🤦🏻

bnb commented 3 years ago

Added that cat command - I need to read up more on that usage of cat since it doesn't seem to have worked precisely how it should have (the build should have failed, but it succeeded 🤔 ).

I could have also put it in the wrong spot.

bnb commented 3 years ago

theoretically the PR linked below should have fixed the issue, I'll see if I can update the PR soon.

https://github.com/tcort/markdown-link-check/pull/106

Trott commented 1 year ago

I've unarchived this repo so I can close all PRs and issues before re-archiving.