Consider supporting OpenGraph?

[malmaud]

As discussed here, OpenGraph support would allow Discourse forums and other popular systems to display inline previews of links to readthedocs pages, especially useful for forums about programming languages and tools.

It just takes a few <meta> tags injected into the rendered HTML, so I don't think it should be too burdensome. Plus Jeff Atwood himself is pushing for it :)

[agjohnson]

Seems like a great addition. It should be trivial to implement on our theme overrides. Does implementing this as an article type make sense, or is there another type that would work best for something like the DIscourse onebox?

[malmaud]

Yes, article type makes sense.

I think "og:description" will be the trickiest tag to get right - it along with "og:title" contain the text which is actually shown in the onebox. According to this, most sites will display between 200 and 300 characters of the og:description value, so it essentially is a 200-character summary of the doc page.

[agjohnson]

Yeah, I'm not entirely sure how to handle the description.

The change for the meta tags is a simple addition to the template overrides RTD maintains. We do have access to the content of the page in the templates, but it's HTML, which rules out a simple truncation operation inside the templates. If we want to be more intelligent about the content truncation, we'll have to write a template filter in a Sphinx extension to break up the HTML content.

For now, this might just be the project description field in the RTD project admin, instead of per-page content for the description.

og:title should be trivial however, this can be provided by the Sphinx template easily.

[malmaud]

That sounds good. And maybe set og:image to the project logo whose name is somehow exposed to the template.

As a concrete reference, here's what the meta tags for say http://docs.julialang.org/en/release-0.4/manual/variables/ could maybe end up looking like:

<meta property="og:title" content="Variables" />
<meta property="og:type" content="article" />
<meta property="og:url" content="http://docs.julialang.org/en/release-0.4/manual/variables/" />
<meta property="og:site_name" content="readthedocs: Julia" />
<meta property="og:description" content="The Julia programming language.." />
<meta property="og:image" content="http://docs.julialang.org/en/release-0.4/_static/julia-logo.svg"/>

[agjohnson]

If we ever start collecting the project logo, that would be another good addition. Currently we aren't collecting that and don't have a great way to determine that outside of project configuration options.

This reference seems like a reasonable place to start at least

[stsewd]

I was able to do a POC here, injecting all the metatags on the rtd theme https://github.com/rtfd/sphinx_rtd_theme.

I was able to make work: title, site_name, url, type, description, image and locale (this one isn't on the same format of language_TERRITORY because of https://www.sphinx-doc.org/en/stable/usage/configuration.html#confval-language).

But, there is a design decision to take here, we depend on the theme_canonical_url value here, but on the rtd code, we inject the canonical url tag from our extension (https://github.com/rtfd/readthedocs-sphinx-ext) rather then injection those values in the html_theme_options or using the value from html_context (we populate this field on the rtd).

So, should this live on the rtd-theme or on the rtd-sphinx-extension?

[stsewd]

I wasn't able to make it work with ngrok, so, I just upload it on github pages, and for some reason the _static dir doesn't work (I guess because of jekyll). But this is kind of how it looks my POC.

this is how it looks without og, I'll need to figureout how to get a better description p:

[stsewd]

I was able to do the .nojekyll thing, but facebook doesn't support svg images p: so, it kind of looks like the same https://developers.facebook.com/tools/debug/sharing/?q=https%3A%2F%2Fstsewd.github.io%2Frtd-test%2F

[stsewd]

At least on twitter it makes a LOT of o difference :grin: https://cards-dev.twitter.com/validator

[humitos]

So, should this live on the rtd-theme or on the rtd-sphinx-extension?

I think that our sphinx extension will be better since it will affect all the themes that are built on Read the Docs, not only our own.

Also, I think that we may sanitize the description probably to make it looks nice (without \n and as a complete sentence). In the example, the title is repeated in the description.

We may want to use the default Read the Docs logo if we don't have a project's image (html_logo in conf.py I suppose).

[agjohnson]

I'd say no logo if the user hasn't specified. It seems odd to brand with our logo

[agjohnson]

So, another thing to point out is that Sphinx already has support for this natively through reST meta directive. It just outputs the meta elements in <head>, but does require authoring these pieces on each page source. Perhaps we can set default page-level meta through standard Sphinx options or and extension?

Ultimately, I want opengraph enabled across all projects. If we are hard coding the template, we'll probably break or at least override anyone using meta for opengraph already. However, setting default meta is probably the cleanest solution either way.

I haven't looked at how sphinx handles this, but my assumption is that the meta elements will be in the doctree after parsing the source, so we can use an extension to add default meta tags based on RTD context data to each document.

http://docutils.sourceforge.net/docs/ref/rst/directives.html#meta

[stsewd]

@agjohnson so, I was editing the extension, with the doctree and everything, but I hit a bug that makes impossible to do the meta tag default thing https://sourceforge.net/p/docutils/bugs/281/, doctutils doesn't support : in the meta directive, even escaping doesn't work as expected.

So, another option is using the builder I guess? I didn't try but, I guess we have access to the dom

[stsewd]

Sphinx issue https://github.com/sphinx-doc/sphinx/issues/2645

[stsewd]

We inject a template already https://github.com/rtfd/readthedocs-sphinx-ext/blob/8e545547c2c1324368406a710dd76e0534674499/readthedocs_ext/readthedocs.py#L117-L121, but I'll try to look if we can have the dom, or block this till the problem is resolved in sphinx

[stsewd]

Another sphinx related issue https://github.com/sphinx-doc/sphinx/issues/6089

[humitos]

doctutils doesn't support : in the meta directive, even escaping doesn't work as expected.

This is already fixed on docutils, but it's not released yet. It will be on 0.15: http://docutils.sourceforge.net/RELEASE-NOTES.html#release-0-15b-dev

[humitos]

docutils 0.15 was released: https://pypi.org/project/docutils/

This is not blocked anymore :tada:

[Daltz333]

Any update on this?

[stsewd]

@Daltz333 sorry, this isn't is our roadmap this month, but I'll try to see if we can prioritize this soon.

[Daltz333]

No rush. It'd be pretty cool though.

[Daltz333]

I went ahead and published https://pypi.org/project/sphinxext-opengraph/

[stsewd]

Just linking to https://github.com/sphinx-doc/sphinx/pull/7974 it may be implemented in sphinx itself. I'll keep this open to see if we can set some values for the user if that happens.

[Daltz333]

In that PR, I've suggested that it may be better to keep it in an external third-party extension, that way we can link straight into READTHEDOCS various environment variables to easily build URLs, instead of having them be static and per language and version.

[stsewd]

So, we already set some variables based on the project from the user, so supporting that wouldn't be hard if the extension is builtin https://github.com/readthedocs/readthedocs.org/blob/f48fb667778a749089cf8adc215a20c4b40e86d3/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl#L150-L150

[Daltz333]

Currently out extension (one I linked previously) will grab language and version through the READTHEDOCS environment variable, that way the user just has to set the base "domain" URL. It would be great if we could grab the domain URL from Readthedocs somehow.

[stsewd]

You can get the canonical url from the context variable (that should be available for plugins), that URL always points to the default version.

https://github.com/readthedocs/readthedocs.org/blob/21e8237940803f4b05ed883386374fcf444b99bd/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl#L94-L94

[Daltz333]

With the 0.4.0 release of sphinxext-opengraph, projects can now just add the extension to their sphinx extensions, and it'll automatically detect and configure when in a RTD environment.

It may be beneficial to the RTD team to inject this for users (add a project flag to test?) in the future.

Our goals for a 1.0.0 release is a bit more extensibility outside of standard opengraph (twitter cards, google language selection). However, the extension has been stable for awhile with a full suite of tests.

[stsewd]

With the 0.4.0 release of sphinxext-opengraph, projects can now just add the extension to their sphinx extensions, and it'll automatically detect and configure when in a RTD environment.

Nice!

It may be beneficial to the RTD team to inject this for users (add a project flag to test?) in the future.

We are moving away from installing things for users (we want to keep build env as close as possible as the one used locally).

We can mention the use of the extension somewhere in the docs.

Also, since this is already implemented, I don't think it makes sense for us to re-implement it, so +1 on just documenting it.

[agjohnson]

Yeah, I'd agree we don't want to install this for users. Documenting this extension would be a good first step.

I'd still like to have this as a feature across all of RTD hosted docs though, not just on some projects. I would leave this issue open because of that.

[Daltz333]

Would this mean RTD would inject into the users HTML output or build? Would this conflict with sphinxext-opengraph or would it be a documentation only approach?

This seems to conflict with @stsewd

[stevepiercy]

@Daltz333 I'm not sure why I was @'ed, as I have not participated in this discussion.

[Daltz333]

My bad, autocorrect on phone github

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[astrojuanlu]

The original request was about supporting OpenGraph for sites hosted on Read the Docs by injecting some magic for our users. Now, sphinxext-opengraph exists, users can adopt it if they want, and we are moving away from injecting extensions for users.

Therefore, I propose we close this issue and open a new one (or discuss internally) on which RTD-owned projects do we want to adopt this extension.

[stsewd]

We should still document this solution in our docs as a guide.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[astrojuanlu]

After our experience with https://github.com/readthedocs/blog/, to support OpenGraph on Read the Docs people need to

1. Add https://pypi.org/project/sphinxext-opengraph/ to their requirements
2. Enable sphinxext.opengraph in their Sphinx extensions

The rest is automatic, since sphinxext-opengraph does some magic if the project is hosted on Read the Docs, as explained in https://github.com/wpilibsuite/sphinxext-opengraph#options.

I think this would be quite short for a guide, but if folks still think it would be useful, I will be happy to write it.

[humitos]

I think this would be quite short for a guide, but if folks still think it would be useful, I will be happy to write it.

@astrojuanlu In case a guide does not quite fit here, I remember that we talked about collecting a collection of Sphinx's extensions and recommend them with a small config file ready to use. If we still want to go in that direction, we could use https://sphinx-extensions.readthedocs.io/ for that.

[astrojuanlu]

We have expressed elsewhere that usually we don't recommend third-party extensions in our docs (see for example discussion at https://github.com/readthedocs/readthedocs.org/pull/8396), so I think we should close this issue and continue the discussion on https://github.com/humitos/sphinx-extensions/issues/14.

On a related note, I think we should give https://sphinx-extensions.readthedocs.io/ more attention and visibility.

[agjohnson]

On a related note, I think we should give https://sphinx-extensions.readthedocs.io/ more attention and visibility.

Yeah I agree, this might make more sense as an official RTD project even

However, if we are pointing users towards another project for this sort of content, seems like something would could also include as narrative content in our own docs. Perhaps guide content is the best avenue for this sort of guidance.

There is a lot of room for us (or Sphinx, but historically us) to highlight how to solve these specific customer/user issues when authoring documentation with Sphinx. Sphinx-extensions helps with illustration, but guide content would probably help translate problem -> solution more for customers/users -- for example, translate "how do I author documentation with multiple code examples" -> "you can use sphinx-tabs, copybutton, ..." etc.

[humitos]

Yeah I agree, this might make more sense as an official RTD project even

I'm happy to move to the readthedocs organization.

However, if we are pointing users towards another project for this sort of content, seems like something would could also include as narrative content in our own docs. Perhaps guide content is the best avenue for this sort of guidance.

When I created the project, I didn't want to generate too much written content due to the lack of time. I'd like to keep each page simple if possible so we can highlight extensions quicker without the need of writing a full guide about it.

[agjohnson]

I'd like to keep each page simple if possible so we can highlight extensions quicker without the need of writing a full guide about it.

Yeah this seems fine, it might be helpful to have two separate overviews of extensions -- narrative, describing why or how authors could solve a particular problem with an extension, and illustrative, showing how each extension functions with the sphinx-extensions repo. The two seem complimentary for now

[humitos]

I just wanted to comment here that sphinx-opengraph is moving forward on supporting social previews (https://github.com/wpilibsuite/sphinxext-opengraph/issues/66#issuecomment-1335066360).

If you are interested in opengraph, you should follow that repository since it's the right extension to install in your documentation.