exercism / problem-specifications

Shared metadata for exercism exercises.
MIT License
327 stars 542 forks source link

Check links in toml and json files #2294

Closed angelikatyborska closed 1 year ago

angelikatyborska commented 1 year ago

We're adding the same link checker to the Elixir repo to catch dead links (PR) and I noticed it's failing on some URLs that come from this repository, three URLs from https://users.csc.calpoly.edu. How could that be? Well, this repository contains exercise metadata in toml files, which contain source URLs, which do not get checked.

Adding the check for toml and json files increases the number of links checked from 98 to 212.

~This is a draft because I'm not sure yet how to fix those three URLs...~ I will just ignore them.


| ✗ [ERR] https://users.csc.calpoly.edu/~jdalbey/103/Projects/ProgrammingPractice.html | Failed: Network error: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1921: (unable to get local issuer certificate)
```~
wolf99 commented 1 year ago

Really great that you've identified this! If I get time I'll investigate also, but otherwise Iooking forward to other's thoughts on it.

wolf99 commented 1 year ago

I see that the elixir track is adding checking of JSON files also. Would it be good to add that here too?

angelikatyborska commented 1 year ago

Yes, absolutely!

What I tried to far: apt update && apt upgrade ca-certificates, but that didn't help :(

angelikatyborska commented 1 year ago

I'll just add that server to ignored links. It looks like it's SSL certificate is misconfigured or something.

The PR can be reviewed, but it requires https://github.com/exercism/problem-specifications/pull/2293 to be merged first to have green CI.

kytrinyx commented 1 year ago

I've rebased onto main.

kytrinyx commented 1 year ago

All the twitter links are timing out right now. I tried re-running the link checking job, but it's still timing out on the twitter links. Let's try rerunning it later to see if it manages to get through it.

angelikatyborska commented 1 year ago

I forgot about this PR 🤦

All Twitter links are still timing out. I stopped using Twitter a while ago, but I think currently not logged in users are only allowed to see pages for single tweets. They are not allowed to see profile pages for example. I'm not allowed to access https://twitter.com/jeg2. And scripts aren't allow to access anything. curl -I on a valid tweet URL always returns 403.

I think the only solution is for us to remove all Twitter links from this repo. People cannot access them anyway.

Unrelated, the web archive is returning 503s today. I hope that's temporary.

ErikSchierboom commented 1 year ago

All Twitter links are still timing out. I stopped using Twitter a while ago, but I think currently not logged in users are only allowed to see pages for single tweets. They are not allowed to see profile pages for example. I'm not allowed to access https://twitter.com/jeg2. And scripts aren't allow to access anything. curl -I on a valid tweet URL always returns 403.

That is absolutely bonkers, but it is true! 🤦

I think the only solution is for us to remove all Twitter links from this repo. People cannot access them anyway.

I agree. We can always restore them if something changes.