[v2] Verify that all stub articles actually exist

[jlengstorf]

There are several docs I've found that are defined in doc-links.yaml as stubs, but that don't have corresponding stub Markdown files. For example, I'm pretty sure none of the stubs under "Sourcing Content and Data" exist.

We need to do a once-over to ensure that the stubs exist to avoid 404s.

Docs that need to be created:

• [x] /docs/docs/unit-testing.md
• [x] /docs/docs/testing-components-with-graphql.md
• [x] /docs/docs/end-to-end-testing.md
• [x] /docs/docs/testing-css-in-js.md
• [x] /docs/docs/testing-react-components.md
• [x] /docs/docs/adding-search.md
• [x] /docs/docs/adding-third-party-services.md
• [x] /docs/docs/adding-analytics.md
• [x] /docs/docs/adding-forms.md

The stub template is here:

https://github.com/gatsbyjs/gatsby/blob/151b1f9a7e1dac890779941db24eff9f35900b8c/docs/docs/authentication-tutorial.md

For anyone who'd like to open a PR, you can add one or more of these stubs in a pull request. This is a great way for first-timers to get involved!

cc: @shannonbux @amberleyromo @kkemple

[amberleyromo]

I'll take care of it 👍🏼

[m-allanson]

Can we run a link checker to make sure we're not missing anything once this is done?

[jlengstorf]

Update robots.txt temporarily so we can run the broken link checker.

[pieh]

https://github.com/stevenvachon/broken-link-checker seems to work locally with current robots.txt currently running it with blc -g -r -f -e https://next.gatsbyjs.org (recursively + only internal links) and it produces:

Getting links from: https://next.gatsbyjs.org/docs/
...
├───OK─── https://next.gatsbyjs.org/docs/post-css/
├───OK─── https://next.gatsbyjs.org/docs/testing/
├─BROKEN─ https://next.gatsbyjs.org/docs/unit-testing/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/testing-components-with-graphql/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/end-to-end-testing/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/testing-css-in-js/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/testing-react-components/ (HTTP_404)
├───OK─── https://next.gatsbyjs.org/docs/debugging/
├───OK─── https://next.gatsbyjs.org/docs/debugging-html-builds/

Just after a while I'm getting some ERRNO_ECONNRESET which I think my connection can't handle all the concurrent requests

[pieh]

All right, I probably tried to do too much - here are broken links from /docs page (with some false negatives because of my connection) - but enough to spot actual broken links

Getting links from: https://next.gatsbyjs.org/docs/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-starters/
├───OK─── https://next.gatsbyjs.org/docs/recipes/
├───OK─── https://next.gatsbyjs.org/docs/overview/
├───OK─── https://next.gatsbyjs.org/docs/preparing-your-environment/
├───OK─── https://next.gatsbyjs.org/docs/basic-hardware-software-requirements/
├───OK─── https://next.gatsbyjs.org/docs/browser-support/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-on-macs/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-on-windows/
├───OK─── https://next.gatsbyjs.org/docs/deploying-and-hosting/
├───OK─── https://next.gatsbyjs.org/docs/preparing-for-deployment/
├───OK─── https://next.gatsbyjs.org/docs/deploy-gatsby/
├───OK─── https://next.gatsbyjs.org/docs/path-prefix/
├───OK─── https://next.gatsbyjs.org/docs/how-gatsby-works-with-github-pages/
├───OK─── https://next.gatsbyjs.org/docs/customization/
├───OK─── https://next.gatsbyjs.org/docs/babel/
├───OK─── https://next.gatsbyjs.org/docs/add-custom-webpack-config/
├───OK─── https://next.gatsbyjs.org/docs/custom-html/
├───OK─── https://next.gatsbyjs.org/docs/environment-variables/
├───OK─── https://next.gatsbyjs.org/docs/api-proxy/
├───OK─── https://next.gatsbyjs.org/docs/images-and-files/
├───OK─── https://next.gatsbyjs.org/docs/adding-images-fonts-files/
├───OK─── https://next.gatsbyjs.org/docs/static-folder/
├───OK─── https://next.gatsbyjs.org/docs/dropping-images-into-static-folders/
├───OK─── https://next.gatsbyjs.org/docs/importing-single-files/
├───OK─── https://next.gatsbyjs.org/docs/importing-media-content/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-source-filesystem-programmatic-import/
├───OK─── https://next.gatsbyjs.org/docs/working-with-images/
├───OK─── https://next.gatsbyjs.org/docs/content-and-data/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-the-filesystem/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-databases/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-saas-services/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-private-apis/
├───OK─── https://next.gatsbyjs.org/docs/headless-cms/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-netlify-cms/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-wordpress/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-drupal/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-contentful/
├───OK─── https://next.gatsbyjs.org/docs/sourcing-from-prose/
├───OK─── https://next.gatsbyjs.org/docs/graphql/
├───OK─── https://next.gatsbyjs.org/docs/introducing-graphiql/
├───OK─── https://next.gatsbyjs.org/docs/creating-and-modifying-pages/
├───OK─── https://next.gatsbyjs.org/docs/page-query/
├───OK─── https://next.gatsbyjs.org/docs/static-query/
├───OK─── https://next.gatsbyjs.org/docs/adding-markdown-pages/
├───OK─── https://next.gatsbyjs.org/docs/adding-a-list-of-markdown-blog-posts/
├───OK─── https://next.gatsbyjs.org/docs/creating-slugs-for-pages/
├───OK─── https://next.gatsbyjs.org/docs/programmatically-create-pages-from-data/
├───OK─── https://next.gatsbyjs.org/docs/plugins/
├───OK─── https://next.gatsbyjs.org/docs/plugin-authoring/
├───OK─── https://next.gatsbyjs.org/docs/submit-to-plugin-library/
├───OK─── https://next.gatsbyjs.org/docs/source-plugin-tutorial/
├───OK─── https://next.gatsbyjs.org/docs/styling/
├───OK─── https://next.gatsbyjs.org/docs/css-modules/
├───OK─── https://next.gatsbyjs.org/docs/typography-js/
├───OK─── https://next.gatsbyjs.org/docs/glamor/
├───OK─── https://next.gatsbyjs.org/docs/styled-components/
├───OK─── https://next.gatsbyjs.org/docs/creating-global-styles/
├───OK─── https://next.gatsbyjs.org/docs/component-css/
├───OK─── https://next.gatsbyjs.org/docs/post-css/
├───OK─── https://next.gatsbyjs.org/docs/testing/
├─BROKEN─ https://next.gatsbyjs.org/docs/unit-testing/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/testing-components-with-graphql/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/end-to-end-testing/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/testing-css-in-js/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/testing-react-components/ (HTTP_404)
├───OK─── https://next.gatsbyjs.org/docs/debugging/
├───OK─── https://next.gatsbyjs.org/docs/debugging-html-builds/
├───OK─── https://next.gatsbyjs.org/docs/debugging-replace-renderer-api/
├───OK─── https://next.gatsbyjs.org/docs/debugging-the-build-process/
├───OK─── https://next.gatsbyjs.org/docs/performance-tracing/
├───OK─── https://next.gatsbyjs.org/docs/building-apps-with-gatsby/
├───OK─── https://next.gatsbyjs.org/docs/progressive-web-app/
├───OK─── https://next.gatsbyjs.org/docs/creating-dynamically-rendered-navigation/
├───OK─── https://next.gatsbyjs.org/docs/authentication/
├───OK─── https://next.gatsbyjs.org/docs/e-commerce/
├───OK─── https://next.gatsbyjs.org/docs/interactive-pages/
├───OK─── https://next.gatsbyjs.org/docs/preparing-for-site-launch/
├───OK─── https://next.gatsbyjs.org/docs/add-404-page/
├─BROKEN─ https://next.gatsbyjs.org/docs/adding-search/ (ERRNO_ECONNRESET)
├───OK─── https://next.gatsbyjs.org/docs/adding-tags-and-categories-to-blog-posts/
├───OK─── https://next.gatsbyjs.org/docs/creating-a-sitemap/
├───OK─── https://next.gatsbyjs.org/docs/linking-between-pages/
├───OK─── https://next.gatsbyjs.org/docs/seo/
├─BROKEN─ https://next.gatsbyjs.org/docs/adding-third-party-services/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/adding-search/ (HTTP_undefined)
├─BROKEN─ https://next.gatsbyjs.org/docs/adding-analytics/ (HTTP_404)
├─BROKEN─ https://next.gatsbyjs.org/docs/adding-forms/ (HTTP_404)
├───OK─── https://next.gatsbyjs.org/docs/performance/
├───OK─── https://next.gatsbyjs.org/docs/caching/
├───OK─── https://next.gatsbyjs.org/docs/local-https/
├───OK─── https://next.gatsbyjs.org/docs/lighthouse-audit/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-link/
├───OK─── https://next.gatsbyjs.org/docs/node-apis/
├───OK─── https://next.gatsbyjs.org/docs/browser-apis/
├───OK─── https://next.gatsbyjs.org/docs/ssr-apis/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-config/
├───OK─── https://next.gatsbyjs.org/docs/actions/
├───OK─── https://next.gatsbyjs.org/docs/graphql-reference/
├───OK─── https://next.gatsbyjs.org/docs/node-interface/
├───OK─── https://next.gatsbyjs.org/docs/api-specification/
├───OK─── https://next.gatsbyjs.org/docs/v2-release-notes/
├───OK─── https://next.gatsbyjs.org/docs/v1-release-notes/
├───OK─── https://next.gatsbyjs.org/docs/v0-release-notes/
├───OK─── https://next.gatsbyjs.org/docs/migrating-from-v1-to-v2/
├───OK─── https://next.gatsbyjs.org/docs/migrating-from-v0-to-v1/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-core-philosophy/
├───OK─── https://next.gatsbyjs.org/docs/life-and-times-of-a-gatsby-build/
├───OK─── https://next.gatsbyjs.org/docs/building-with-components/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-lifecycle-apis/
├───OK─── https://next.gatsbyjs.org/docs/prpl-pattern/
├───OK─── https://next.gatsbyjs.org/docs/querying-with-graphql/
├───OK─── https://next.gatsbyjs.org/docs/authentication-tutorial/
├───OK─── https://next.gatsbyjs.org/docs/ecommerce-tutorial/
├───OK─── https://next.gatsbyjs.org/docs/wordpress-source-plugin-tutorial/
├───OK─── https://next.gatsbyjs.org/docs/image-tutorial/
├───OK─── https://next.gatsbyjs.org/docs/community/
├───OK─── https://next.gatsbyjs.org/docs/code-of-conduct/
├───OK─── https://next.gatsbyjs.org/docs/how-to-contribute/
├───OK─── https://next.gatsbyjs.org/docs/contributor-swag/
├───OK─── https://next.gatsbyjs.org/docs/how-to-file-an-issue/
├───OK─── https://next.gatsbyjs.org/docs/how-to-label-an-issue/
├───OK─── https://next.gatsbyjs.org/docs/how-to-submit-a-pr/
├───OK─── https://next.gatsbyjs.org/docs/pair-programming/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-style-guide/
├───OK─── https://next.gatsbyjs.org/docs/design-principles/
├───OK─── https://next.gatsbyjs.org/docs/rfc-process/
├───OK─── https://next.gatsbyjs.org/docs/templates/
├───OK─── https://next.gatsbyjs.org/docs/how-to-run-a-gatsby-workshop/
├───OK─── https://next.gatsbyjs.org/docs/how-to-pitch-gatsby/
├───OK─── https://next.gatsbyjs.org/docs/submit-a-blogpost/
├───OK─── https://next.gatsbyjs.org/docs/site-showcase-submissions/
├───OK─── https://next.gatsbyjs.org/docs/submit-to-starter-showcase/
├───OK─── https://next.gatsbyjs.org/docs/glossary/
├───OK─── https://next.gatsbyjs.org/docs/gatsby-cli/
Finished! 189 links found. 55 excluded. 10 broken.

[jlengstorf]

Great! Thanks for this, @pieh!

[amberleyromo]

It took 30 minutes, but I got the full site output from that tool, if anyone is interested.

https://gist.github.com/amberleyromo/78ad2efaabeac53208626a16b8b7e047

I have what I need for the docs work though. Thanks y'all

[kartiklad]

@amberleyromo Seems like the list has a lot of Broken links. Happy to help if you need! :)

[jlengstorf]

We'd love to have your help, @kartiklad! I'm not sure that everything in @amberleyromo's report is actually broken; I think that tool sent back a lot of false positives. The checklist at the top of this issue is a better starting point.

Thanks so much for contributing! 💪💜

[kartiklad]

Awesome! I’ll get on it and send a PR soon...cheers!

[shannonbux]

These ones should all work now and I've updated the checklist at the top of the issue:

├─BROKEN─ https://next.gatsbyjs.org/docs/unit-testing/ (HTTP_404) ├─BROKEN─ https://next.gatsbyjs.org/docs/testing-components-with-graphql/ (HTTP_404) ├─BROKEN─ https://next.gatsbyjs.org/docs/end-to-end-testing/ (HTTP_404) ├─BROKEN─ https://next.gatsbyjs.org/docs/testing-css-in-js/ (HTTP_404) ├─BROKEN─ https://next.gatsbyjs.org/docs/testing-react-components/ (HTTP_404)

On Fri, Aug 17, 2018 at 1:39 AM, Kartik Lad notifications@github.com wrote:

Awesome! I’ll get on it and send a PR soon...cheers!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gatsbyjs/gatsby/issues/7294#issuecomment-413784249, or mute the thread https://github.com/notifications/unsubscribe-auth/Ae9o2nER0i0CcdthEZlsmPrcB1mW-0daks5uRnMjgaJpZM4V7dsW .

[shannonbux]

Looks like the "Adding Third Party Services" overview and its children are the only things broken. Also, "Adding Search" exists, but in a different section of the docs! We need to just move it to "Adding Third Party Services", I suppose. I'm not sure if that's the right choice.

@jlengstorf, the "Sourcing content and data" section works for me. Does it still not work for you?

Also, wondering if we ought to rename https://next.gatsbyjs.org/docs/headless-cms/ to "Sourcing from Headless CMSs" <--does that pluralization bother anyone else on a nav bar? It's not my favorite. We could do "Sourcing from a Headless CMS". Once we agree on whether/which title change to do, I can do it and then add a redirect.

[kartiklad]

@shannonbux I think "Sourcing from a Headless CMS" sounds better

[kartiklad]

Hey @shannonbux @jlengstorf,

Just made a pull request(#7432) to add the missing pages and also moved the Adding search page to 3rd party services. Hope it's ok?

[jlengstorf]

Looks great, @kartiklad — merged! Thanks so much for your help! 💪💜