Github action that checks links

[cam-barts]

I heard you talking about this project on Talk Python to Me and you mentioned how much of a hassle it is to check all the links for 404s, and it inspired me to create a GitHub action to help with the process. It definitely could still use work, but I thought it was a start and would help weed out the known 200s.

[cam-barts]

Example output

[mahmoud]

Ohhh very interesting! That's wonderful!

So here're the ways I can think of to improve it, though I don't know when I'll have time to do so:

• Just check the projects.yaml file
• Can we generate a structured report that's somewhat stateful? Like, sometimes links are temporarily down, and it only makes sense to review/remove them when they've been down for a while.
• Do we want to run this on every push? I get a little worried about bombarding some of the littler sites out there. Then again, how much traffic could we be generating?

Thanks again!

[cam-barts]

Thanks for the feedback! I have some thoughts that you might consider.

• Is the projects.yaml file unique to this project? I'd like to make the action useful for a broader audience, and I don't immediately see the benefit of checking the yaml over the markdown when a broader audience uses markdown exclusively.
• This is my first real attempt at making an action, so I am not completely versed in it, but I am sure there would be a way of creating something more stateful. My immediate idea would be to open an issue with the report, and have the action update the issue. I think for that approach, you would need to pass a github token. I know the pycodestyle action leaves comments on PRs, but for something like a dead link monitor it might be better to have as an issue. What do you think?
• Assuming there aren't very many links that go to the same domain, the load this would bring to a site at worst would be the (# pushes + 1 requests)/hour and at least 1 request/hour. Just being on an Awesome List generates traffic, but I can see your point. Some ideas that I have for this is maybe allow the list maintainer to set a custom user agent string, so the site knows at least where the traffic is coming from. Then, they can outright block the requests if they believe it is abusive, or (in a future iteration of the action with an allow list) they can submit a PR to get on the action's allow list. What do you think about that?

[mahmoud]

That all sounds fair enough. And this is definitely an improvement over what I had before. Is there a way you could make the action take an argument of which files to check?

[cam-barts]

@mahmoud Yea man, I am glad you like it. This is just the first stage, the next features I will add based on your feedback are probably going to be stateful reports and I was thinking more of an inclusion/exclusion list for the links. It would take much to add some functionality to include or exclude files, so I will work that in there, just as soon as some time fills up on my calendar to work on it. I am excited that you like it, and that it seems like it will be useful for you!

[mahmoud]

I figure it's about time this was merged, take it for a spin in the real world here. Keep us posted on updates, and thanks again!