search

mozilla / experimenter

A web application for managing user experiments for Mozilla Firefox.
https://experimenter.services.mozilla.com
Mozilla Public License 2.0
120 stars 186 forks source link

Add Markdown link checker? #4845

Closed pdehaan closed 3 years ago

pdehaan commented 3 years ago

OK, possibly final one...

I have mixed feelings about this one, but we can try using markdown-link-check. It doesn't handle file globs, so we'll need to use the API and roll our own logic. But meh.

# Will lint specific files A-OK!
npx markdown-link-check docs/jetstream/jetstream.md

FILE: docs/jetstream/jetstream.md
[✓] https://github.com/mozilla/jetstream
[✓] https://github.com/mozilla/jetstream-config
[✓] https://docs.telemetry.mozilla.org/datasets/jetstream.html

3 links checked.

# When linting via a "glob" as a param, it chooses the first file and quits.
npx markdown-link-check docs/*.md

FILE: docs/data-scientists-root.md
No hyperlinks found!

0 links checked.

We might be able to be able to find a different link checker tool that will scrape an entire HTML site vs the input .md files.

For example, broken-link-checker will check a whole site, but fails on the RemoteSettings pages since they require VPN access. I'm looking through the README, but it still isn't clear if it's possible to add a list of domains/URLs to ignore. (it's the --exclude flag).

npx broken-link-checker https://mozilla.github.io/experimenter-docs/ -ro --exclude="settings-writer.stage.mozaws.net" --exclude="settings-writer.prod.mozaws.net" --filter-level=3
Getting links from: https://mozilla.github.io/experimenter-docs/
├───OK─── https://mozilla.github.io/experimenter-docs/blog/rss.xml
├───OK─── https://mozilla.github.io/experimenter-docs/blog/atom.xml
├───OK─── https://mozilla.github.io/experimenter-docs/opensearch.xml
├───OK─── https://mozilla.github.io/experimenter-docs/img/favicon.ico
├───OK─── https://bh4d9od16a-dsn.algolia.net/
├─BROKEN─ https://github.com/mozilla/experimenter-docs/experimenter-docs/ (HTTP_404)
├───OK─── https://mozilla.github.io/experimenter-docs/styles.faaec3e4.css
├───OK─── https://mozilla.github.io/experimenter-docs/styles.02f31503.js
├───OK─── https://mozilla.github.io/experimenter-docs/runtime~main.401454fb.js
├───OK─── https://mozilla.github.io/experimenter-docs/main.582e1fdd.js
├───OK─── https://mozilla.github.io/experimenter-docs/common.04ab76a4.js
├───OK─── https://mozilla.github.io/experimenter-docs/2.5da1d098.js
├───OK─── https://mozilla.github.io/experimenter-docs/13.7f028b15.js
├───OK─── https://mozilla.github.io/experimenter-docs/16.ae14cd00.js
├───OK─── https://mozilla.github.io/experimenter-docs/935f2afb.88c1f4ed.js
├───OK─── https://mozilla.github.io/experimenter-docs/17896441.24042e66.js
├───OK─── https://mozilla.github.io/experimenter-docs/ebfee794.9af5c413.js
├───OK─── https://v2.docusaurus.io/docs/docusaurus.config.js/#baseurl
├───OK─── https://mozilla.github.io/experimenter-docs/img/logo.svg
├───OK─── https://mozilla.github.io/experimenter-docs/img/logo-dark.svg
├───OK─── https://experimenter.services.mozilla.com/nimbus/
├───OK─── https://mana.mozilla.org/wiki/display/FJT/Project+Nimbus
├───OK─── https://github.com/mozilla/experimenter-docs
├───OK─── https://mana.mozilla.org/wiki/display/FJT/Nimbus+Onboarding#NimbusOnboarding-Branches
├───OK─── https://mana.mozilla.org/wiki/display/FJT/How+to+Launch+a+Nimbus+Experiment
├───OK─── https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=130920248
├───OK─── https://mana.mozilla.org/wiki/display/FJT/Nimbus+Engineering#NimbusEngineering-ExperimentTelemetry
├───OK─── https://mana.mozilla.org/wiki/display/FJT/Lessons+Learned+-+Experiment+Incidents+and+Close+Calls
├───OK─── https://github.com/mozilla/jetstream
├───OK─── https://docs.telemetry.mozilla.org/datasets/jetstream.html
├───OK─── https://mozilla.github.io/experimenter-docs/experiment-owners
├───OK─── https://mozilla.github.io/experimenter-docs/fenix-engineers
├───OK─── https://docs.google.com/document/d/1kchihPal0__A4VOAiPJarNuZXns5KhOHHfIeIzT6zfU/edit#
├───OK─── https://mozilla.github.io/experimenter-docs/desktop-engineers
├───OK─── https://www.notion.so/Nimbus-Migration-Guide-for-Desktop-Front-End-Experiments-d36c21e505f84550aad1202897fc4ba3
├───OK─── https://docs.google.com/document/d/1ev75pG0nAM1lz53WuPQkWqykUlZMmZRbx8wzvvn5DhU/edit
├───OK─── https://mozilla.github.io/experimenter-docs/data-scientists
├───OK─── https://github.com/mozilla/experimenter
├───OK─── https://v2.docusaurus.io/
├───OK─── https://experimenter.services.mozilla.com/
├───OK─── https://stage.experimenter.nonprod.dataops.mozgcp.net/
├───OK─── https://experimenter.services.mozilla.com/nimbus
├───OK─── https://stage.experimenter.nonprod.dataops.mozgcp.net/nimbus
├───OK─── https://storage.googleapis.com/mozilla-storybooks-experimenter/index.html
├───OK─── https://github.github.com/gfm/
├───OK─── https://v2.docusaurus.io/docs/markdown-features
├───OK─── https://github.com/mozilla/experimenter-docs/new/main/docs
├───OK─── https://github.com/mozilla/experimenter-docs/blob/main/sidebars.js
├───OK─── https://github.com/mozilla/experimenter-docs/edit/main/docs/welcome.md
Finished! 85 links found. 36 excluded. 1 broken.

Getting links from: https://mozilla.github.io/experimenter-docs/experiment-owners
├─BROKEN─ https://github.com/mozilla/experimenter-docs/experimenter-docs/experiment-owners (HTTP_404)
├───OK─── https://mozilla.github.io/experimenter-docs/e19d52b8.3f571014.js
├───OK─── https://github.com/mozilla/experimenter-docs/edit/main/docs/experiment-owners-root.md
Finished! 63 links found. 60 excluded. 1 broken.

Getting links from: https://mozilla.github.io/experimenter-docs/fenix-engineers
├─BROKEN─ https://github.com/mozilla/experimenter-docs/experimenter-docs/fenix-engineers (HTTP_404)
├───OK─── https://mozilla.github.io/experimenter-docs/f6be3a8e.7f4c1802.js
├───OK─── https://github.com/mozilla/experimenter-docs/edit/main/docs/fenix-engineers-root.md
Finished! 63 links found. 60 excluded. 1 broken.

Getting links from: https://mozilla.github.io/experimenter-docs/desktop-engineers
├─BROKEN─ https://github.com/mozilla/experimenter-docs/experimenter-docs/desktop-engineers (HTTP_404)
├───OK─── https://mozilla.github.io/experimenter-docs/028aca45.8eb2e77a.js
├───OK─── https://github.com/mozilla/experimenter-docs/edit/main/docs/desktop-engineers-root.md
Finished! 63 links found. 60 excluded. 1 broken.

Getting links from: https://mozilla.github.io/experimenter-docs/data-scientists
├─BROKEN─ https://github.com/mozilla/experimenter-docs/experimenter-docs/data-scientists (HTTP_404)
├───OK─── https://mozilla.github.io/experimenter-docs/ac25fb4e.fdee33aa.js
├───OK─── https://github.com/mozilla/experimenter-docs/edit/main/docs/data-scientists-root.md
Finished! 62 links found. 59 excluded. 1 broken.

Finished! 336 links found. 275 excluded. 5 broken.
Elapsed time: 13 seconds
npm ERR! code 1

┆Issue is synchronized with this Jira Task ┆Issue Number: EXP-1084

pdehaan commented 3 years ago

The current broken links might be coming from the <head> metadata (for "canonical" and "og:url" hrefs):

<link rel="canonical" href="https://github.com/mozilla/experimenter-docs/experimenter-docs/" data-react-helmet="true">
<meta property="og:url" content="https://github.com/mozilla/experimenter-docs/experimenter-docs/" data-react-helmet="true">

Note the duplicated "/experimenter-docs/" in the URL, plus... not sure either of those are the canonical URLs (vs the desired https://mozilla.github.io/experimenter-docs/ href, but 🤷 ).