catalyst / moodle-tool_crawler

A moodle link crawling robot, find broken, slow and oversized links
https://moodle.org/plugins/tool_crawler
10 stars 17 forks source link

Add a report showing links to non secure pages both internal and external #1

Open brendanheywood opened 8 years ago

abias commented 4 years ago

Currently, the crawler does not care about the security of the crawled pages, i.e. it does ignore invalid certificates (see https://github.com/catalyst/moodle-tool_crawler/blob/master/classes/robot/crawler.php#L1073-L1074). I would imagine that a site with an invalid certificate should also show up as broken link.

@brendanheywood , is this what you meant with this issue or do you want to highlight HTTP-only links instead in some way?

brendanheywood commented 4 years ago

For internal links I think the original intent of this was to help weed out http links when you had done a site migration. There is a blackboard tool for this which is more appropriate which also fixes the links.

For external links, I think we just want to show non https links and encourage people to avoid using them. Sometimes the external page will redirect to https and that would get picked up in the report which shows redirects, but in some cases an external site would dual serve on http and https without redirecting. It would be nice if we tested that the https version existed, and perhaps asserted that it was similar to the http version (maybe the same title tag), and then suggested that it be used instead. Sort of a pre-emptive opportunistic HSTS