Make OCDS documentation more portable

If we switch to DocSearch https://github.com/open-contracting/ocds-index/issues/7, the deployment process would then just be copying the built pages to the server. We still need our own hosting for:

[ ] Top-level static files like https://standard.open-contracting.org/schema/ If hosted on ReadTheDocs, I think we'd have to use redirects, or release these separately.
[ ] Reverse proxy for Data Review Tool. Though, we can move that to a subdomain, and redirect.
[ ] Banners. Though, we can use JS instead to add an element to the page based on the URL. If hosted on ReadTheDocs, banners for staging versions and old versions are natively supported.
[ ] Robots.txt excludes. Though, we might be able to use a meta tag instead. If hosted on ReadTheDocs, the robots.txt file from the default branch is used. ReadTheDocs also makes it easy to hide old versions.
[ ] Subprojects. If hosted on ReadTheDocs, subprojects can be used. This allows searching across subprojects. However, we'd have a /projects/ path component instead of /profiles/.
[ ] Version switcher https://github.com/pydata/pydata-sphinx-theme/issues/23. If hosted on ReadTheDocs, this is natively supported, though not sure how this works for custom themes.
[ ] Language switcher https://github.com/pydata/pydata-sphinx-theme/pull/360 If hosted on ReadTheDocs, this can be done using multiple projects
[ ] Redirects. Without our own server, we can do this for pages within Sphinx' scope using e.g. https://tech.signavio.com/2017/managing-sphinx-redirects. ReadTheDocs allows manual, online configuration of redirects. Need to check whether other redirects are in fact accessed.
[x] Legacy documentation and Open Data Comparison. Though, it's not clear that we need to keep these online.

We currently need our own build system for:

[ ] current_lang logic for linking to translated schema and codelists. OCDS for PPPs doesn't use it though - in conf.py, it instead writes these files to both the language's root and to a _static path that is ignored by git.
[ ] Compiling non-Sphinx catalogs. We can maybe add some code in conf.py if we don't find another way.
[ ] util/add_translation_notes.py. We should hook this into Sphinx using the build-finished event instead: https://www.sphinx-doc.org/en/master/extdev/appapi.html#sphinx-core-events

If we address all the above, it could be possible to host on ReadTheDocs or elsewhere. To opt-out of ads, see https://readthedocs.org/accounts/gold/

For older instructions on setting up ReadTheDocs: https://ocds-standard-development-handbook.readthedocs.io/en/latest/profiles/technical/integrations.html

If using ReadTheDocs, we should configure:

Notifications for build failures: https://docs.readthedocs.io/en/stable/guides/build-notifications.html#enabling-build-notifications
Canonical URL: https://docs.readthedocs.io/en/stable/guides/canonical.html
Use a configuration file, instead of relying on web configuration: https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html#using-a-configuration-file
Fail on warnings: https://docs.readthedocs.io/en/stable/config-file/v2.html#sphinx-fail-on-warning

We can also eliminate test_common.py as we can trust that the build, search and language switcher work. (Note: When we switch to sphinx-inline-tabs instead of our custom jsoninclude JS, we can also eliminate test_docs.py.)

I think the main question is how well ReadTheDocs' features are supported when using a custom theme. Docs for integration with:

Banners: RTD injects an admonition as HTML, e.g. https://csvkit.readthedocs.io/en/0.9.0/ We should test it with MyST.
Search: https://readthedocs-sphinx-search.readthedocs.io/en/latest/customization.html#custom-search-input but https://github.com/pydata/pydata-sphinx-theme/issues/202
Version/locale flyout: Works, but can use improvement: https://github.com/pydata/pydata-sphinx-theme/issues/194 https://github.com/pydata/pydata-sphinx-theme/issues/327 See also https://github.com/readthedocs/readthedocs.org/issues/7748

Note that the version/locale switchers need to be updated for the new theme anyway https://github.com/open-contracting/standard/pull/1254#issuecomment-822971743. If we decide to switch to RTD, we can avoid doing this work twice.

Hi 👋🏼 . I found this issue while browsing for Read the Docs issues and I wanted to comment some things 😄

Top-level static files like standard.open-contracting.org/schema If hosted on ReadTheDocs, I think we'd have to use redirects, or release these separately.

Subprojects. If hosted on ReadTheDocs, subprojects can be used. This allows searching across subprojects. However, we'd have a /projects/ path component instead of /profiles/.

This limitation was removed and that /projects/ prefix can be changed now.

Version switcher Add version dropdown (for other versions of the docs) pydata/pydata-sphinx-theme#23. If hosted on ReadTheDocs, this is natively supported, though not sure how this works for custom themes.

There is a completely new beta approach to this version switcher that's called "addons". Currently, the version switcher is added at the bottom-right of the page. You can see an example at https://test-builds.readthedocs.io/en/latest/

Also, we are working on a way to heavily customize the version switcher from the Sphinx theme itself by subscribing to a JavaScript CustomEvent and manipulate the Read the Docs data as wanted. You can see an example of this implementation at https://github.com/readthedocs/sphinx_rtd_theme/pull/1526. I'd love your feedback about this approach in case you are considering integrating the version switcher in your custom theme.

Compiling non-Sphinx catalogs. We can maybe add some code in conf.py if we don't find another way.

I understand you need to run custom commands for non-Sphinx catalogs. Read the Docs now supports this by using build.jobs or build.commands config keys, depending on your needs. Documentation at https://docs.readthedocs.io/en/latest/build-customization.html

Banners: RTD injects an admonition as HTML, e.g. csvkit.readthedocs.io/en/0.9.0 We should test it with MyST.

This was also rewritten with the new addons approach and they work on any Sphinx theme, even on different documentation tools, like Docusaurus: https://test-builds.readthedocs.io/en/docusaurus/

Search: readthedocs-sphinx-search.readthedocs.io/en/latest/customization.html#custom-search-input but Support readthedocs-sphinx-search pydata/pydata-sphinx-theme#202

The readthedocs-sphinx-search extension is deprecated in favor of the new addons approach as well 😄 . Now, if you click on the input from the flyout out shown at the bottom-right you will get a new and better version of the "search as you type" feature.

If you want to trigger the "Read the Docs Search addon" when clicking on any other input (e.g. the default search input that Sphinx uses) you can add a small JavaScript code to trigger the correct event on input.focus().

Version/locale flyout: Works, but can use improvement: ReadTheDocs version selector always on top for mobile displays pydata/pydata-sphinx-theme#194 How to activate the RTD flyout menu? pydata/pydata-sphinx-theme#327 See also Make our documentation page APIs more user friendly readthedocs/readthedocs.org#7748

As I mentioned before, we are working on giving the theme authors access to all the data in a JSON structure so they can integrate nicely with the theme as shown in this PR: https://github.com/readthedocs/sphinx_rtd_theme/pull/1526. Again, feedback welcome.

Let me know if this helps you with the decision about whether or not using Read the Docs for your docs 🎉

open-contracting / deploy

Make OCDS documentation more portable #266