python / docs-community

Community management for documentation contributors and the Docs Workgroup

https://docs-community.readthedocs.io/en/latest/

Creative Commons Zero v1.0 Universal

45 stars 25 forks source link

Research ways to improve SEO #63

Open Mariatta opened 2 years ago

Mariatta commented 2 years ago

How to improve docs.python.org's SEO ranking?

Come up with action items on how the SEO can be improved.

Mariatta commented 2 years ago

I did a quick reading on a few blogs about SEO. Here are some things I learned:

Analyze with Google search console https://search.google.com/search-console
Add meta tags https://ahrefs.com/blog/seo-meta-tags/
Fix broken links
Optimize site for mobile devices
Encourage sharing on social media
Write click-worthy titles

hugovk commented 2 years ago

Google Chrome has something in devtools called Lighthouse that runs an audit on a webpage:

Right click page > Inspect > Lighthouse > can uncheck the non-SEO categories for speed > Analyze page load

There's also an online version: https://web.dev/measure/

Results for an arbitrary page (https://docs.python.org/3/library/csv.html) are pretty good to begin with:

https://web.dev/measure/?url=https%3A%2F%2Fdocs.python.org%2F3%2Flibrary%2Fcsv.html

Links to those recommendations:

The first one should be pretty easy, and we should the second. The last is bigger theme work, and we may get a good enough bump from the first one or two.

hugovk commented 2 years ago

For the first one, https://pypi.org/project/sphinxext-opengraph/ is a Sphinx extension to automatically add OpenGraph metadata.

It doesn't add <meta name="description"> (although there's a feature request: https://github.com/wpilibsuite/sphinxext-opengraph/issues/65), but it would be good to add this anyway and could also help with SEO. (And it improves the previews when sharing on Facebook.)

However, we will need to know the version number of the actual docs build (e.g. (it could be /3/, /3.9/ or /3.9.2/. See https://github.com/python/docsbuild-scripts/issues/137

ezio-melotti commented 2 years ago

During the last call someone mentioned that "searching for specific functions doesn't bring docs.p.o up as first result".

In the chat I suggested to add the entries that recently got added to the docs ToC in a <meta name="keyword">, but I'm not sure whether that would belong to built-in Sphinx or if it would be better as an extension (@AA-Turner suggested to create a Sphinx issue, so I could start from there).

However It's also been a few years since I last looked into SEO-related stuff, and I have the impression that nowadays the meta keywords are not as important anymore (I might be mistaken, or there might be something equivalent/better).

Some concern was also brought up about search engines indexing version of the docs that shouldn't be indexed, e.g. docs PR previews or outdated versions. This could be solved by adding an explicit <meta robots="noindex; nofollow">, especially to the former group -- Google got much better with the search result not including outdated versions, and there might be old deprecated/removed features that should still be indexed.

hugovk commented 2 years ago

Good suggestions.

However It's also been a few years since I last looked into SEO-related stuff, and I have the impression that nowadays the meta keywords are not as important anymore (I might be mistaken, or there might be something equivalent/better).

Maybe, although I think a good rule of thumb for good Google results is to follow the suggestions by Google's SEO tool :)

hugovk commented 2 years ago

For the first one, pypi.org/project/sphinxext-opengraph is a Sphinx extension to automatically add OpenGraph metadata.

It doesn't add <meta name="description"> (although there's a feature request: wpilibsuite/sphinxext-opengraph#65), but it would be good to add this anyway and could also help with SEO. (And it improves the previews when sharing on Facebook.)

I contributed some improvements to this extension, including adding <meta name="description">.

We've added it to the devguide, and the SEO score has increased from 91 to 100. It also improves the previews on Facebook, Slack, Discord, etc.

We can look into adding it to the main docs next.

hugovk commented 2 years ago

Next up! PR to add the same thing to https://docs.python.org:

https://github.com/python/cpython/pull/99931

hugovk commented 1 year ago

This is now on /3/ and /3.12/! 🚀

For example:

https://docs.python.org/3/library/csv.html (3.11 branch)
https://docs.python.org/3.12/library/csv.html (main branch)

(Note: It's not on the index.html or download.html pages, because they're html_additional_pages and not under control of the Sphinx extension.)

Google's Pagespeed Insights, SEO score

Increased from 81 to 88 on mobile, and 83 to 92 on desktop. As expected the meta description now passes. ✅

Details

Old /3.10/	New /3/
SEO mobile: 81, desktop: 83	SEO mobile: 88, desktop: 92

Facebook

Here's how the new page looks on Facebook's sharing debugger:

Details

Interestingly, because old pages like https://docs.python.org/3.10/library/csv.html declare their canonical URL to be the /3/ version, we also get that metadata for free!

<link rel="canonical" href="https://docs.python.org/3/library/csv.html" />

So here's 3.10:

Details

We have to go back to 2.7 to see how it was before:

Details

Discord

/3/ is expanded:

Slack

Also /3/ is expanded: