LinkChecker Update

[cam-barts]

I've made some changes to the linkchecker action to address it failing all the time. Here's a summary:

• [x] Added an optional exclusion list file. The app will look for a file in the root of the repository called "exclude_links.json". This will be a json file that has links that are formatted like this:

  {"./BY_PLATFORM.md": {
  "https://pypi.org/project/beancount": {
      "code": "Error (Likely 404)", 
      "time": 1587403869.173222, 
      "reason": ""}, 
  "http://readthedocs.org/docs/rapidsms": {
      "code": 404, 
      "time": 1587403869.17323, 
      "reason": ""}
  }

  The action will produce this json file as an artifact, so you can just add it to the root of the repo. If a link is in the exclusion list, it won't be checked. I left a slot for a reason a link is on the list, in case you wanted to be explicit in why your excluding it. I included epoch time for the last time the link was checked, but that does not have any functionality in the action itself yet. Later, I am hoping to be able to pass some params in to check links that havent been checked in X amount of time.

• [x] Made the checking of the links async. This was just to try to speed things up. Didn't actually test the speed increases, but I will be in the future.
• [x] Action only fails if there are links that failed that aren't in the exclusion list This is the major update in the functionality of the app.

I marked this PR with WIP because I'd love your feedback on the functionality. I mentioned the params change in the future, and I am also open to looking for the exclude_links.json being somewhere that might make some more sense (like the .github folder might be better). Since you are the first one to consume this action, I'd really love to know how you think I could make this better for you, and hopefully for future awesome lists.

[mahmoud]

Hey! This is great!

So my first thought was that, yes, this is best in the .github folder.

Second thought, should "code" also accept a list, or maybe a wildcard? Or is empty/null equivalent to wildcard? I also think it'd be great if the time could be ISO formatted, for readability. datetime.isoformat(), basically (I usually drop the microseconds, but that's just me :) ).

My other thoughts aren't really specific to this PR, I think it's a pretty strong improvement, I'd start using it as soon as the .github folder is in the mix.

That said, if we're adding a .github config though... I still think it'd be nifty to specify which file(s) in the repo to check. This would not only improve output, but probably double the perf, as the APA repeats the links in multiple files. As such, I was going to recommend a more unified and extensible config. .github/linkchecker_config.json, with a key for ignored_links or ignored_errors.

Great work again, and thanks for following up!

[mahmoud]

Hey @cam-barts! How's it going? Any luck?

[cam-barts]

Hey @mahmoud, haven't had a whole lot of time at my desk recently, but it should be done by Wednesday if that's alright

[mahmoud]

Of course! I'll check back Wed/Thurs :)

[cam-barts]

Alrighty, this time around I've:

• [x] Moved the exclude_links to the .github folder
• [x] Changed the time to isoformat
• [x] Added file inclusion/exclusion to the actions environmental variables

[mahmoud]

Nice! So, I still see [WIP] on the ticket name, do you intend to do more on this ticket or should we take it out for a spin?

[cam-barts]

There are still some housekeeping changes I was going to make to the actual linkchecker app, and I am still open to more feedback, but I think its fine to go for now

[mahmoud]

If it's fine to go, I'll give it a go! Thanks, and always looking forward to the next!