Coordinating with RtD about upcoming changes

pradyunsg commented 2 months ago

What's happening?

Read the Docs is going to make changes to both how they're building Sphinx projects as well as how they're injecting functionality into the rendered documentation sets.

https://github.com/readthedocs/addons is the upcoming JS client to enable this functionality, which would be injected server-side when serving HTML pages AFAICT.

Reproducer

N/A

Expectation

This issue would be resolved with Furo making changes to all the spots where it depends on some details related to RtD. As of 2024.04.27, these are:

Code of Conduct

[X] I agree to follow the Code of Conduct.

pradyunsg commented 2 months ago

See #785 where @humitos reached out about these upcoming changes on their end.

pradyunsg commented 2 months ago

Hmm... I think I might need to know the details of how things work on the client-side (https://github.com/readthedocs/addons/issues/59) to make progress here.

humitos commented 2 months ago

I wasn't able to document the process in a clear way yet, but I did the implementation of what I think is a good pattern to follow using our own theme at https://github.com/readthedocs/sphinx_rtd_theme/pull/1526. I'd appreciate any feedback you may have about it.

I did the same for CPython at https://github.com/python/cpython/pull/116966 that you can also take a look. The usage of the data is the same, but it changes how the flyout/selectors are generated. Let me know. I'm happy to help here.

pradyunsg commented 2 months ago

Hmm... http://docs.devthedocs.org/_/addons/?project-slug=docs&version-slug=latest&client-version=0.12.0&api-version=1 from description of https://github.com/readthedocs/sphinx_rtd_theme/pull/1526 is a 404 for me.

Just to make sure my understanding correct:

The deprecation is upcoming (i.e. no changes have been made so far to how projects are built / pages are served).
- The new client is NOT being injected into pages today, when the documentation is being served.
Once the migration is complete, the final state will be:
- No build-time injection, tweaks or Sphinx extensions. Only the various environment variables would be set.
- Client-side injection of the new addons JS file, which would provide the relevant data to the theme. The theme is expected to handle injecting the information as appropriate.

If that understanding is correct, then I have the following questions:

Is it feasible to have a transition like the following?

RtD Functionality Now Now + t1 Now + t2 Now + t3

Existing behaviours 🟢 🟢 🟠 🔴

New behaviours 🔴 🟢 🟢 🟢

🟢: Functional for RtD projects built during this period. 🟠: Deprecated, functional for RtD projects built during this period. 🔴: Non-functional.

(this would mean that I could just wait until the new client is being injected-when-serving in all newly-built docs and also would have confidence that any new RtD build would work correctly)
Is there any way for the theme to check whether the new functionality is going to be enabled / the new client would be injected etc during the build process?
Could you help me figure out what would be equivalent to the things I'm doing in Furo's conf.py gated behind a dev-only conditional + some static files, which functionally is trying to inject all the pieces that RtD's current build-process injection would inject?

https://github.com/pradyunsg/furo/blob/750fcd77fdbd4621919e461d366568847d378f6e/docs/conf.py#L113-L132

https://github.com/pradyunsg/furo/blob/750fcd77fdbd4621919e461d366568847d378f6e/docs/_static/readthedocs-dummy.js#L1-L9

Ideally, I'd prefer to not need to change browser configuration to make things work (I need to test against multiple browsers on multiple platforms somewhat regularly, and the current workflow involves doing so with a network-local server that is accessed from different machines on the network 😅).

RtD Functionality	Now	Now + t1	Now + t2	Now + t3
Existing behaviours	🟢	🟢	🟠	🔴
New behaviours	🔴	🟢	🟢	🟢

pradyunsg commented 2 months ago

Oh, and thanks for reaching out! It's appreciated! ^>^

humitos commented 2 months ago

Hmm... http://docs.devthedocs.org/_/addons/?project-slug=docs&version-slug=latest&client-version=0.12.0&api-version=1 from description of readthedocs/sphinx_rtd_theme#1526 is a 404 for me.

Ups... That's a development instance URL. The one from production is https://docs.readthedocs.io/_/addons/?project-slug=docs&version-slug=latest&client-version=0.12.0&api-version=1

We've already made the changes for this pattern. Following your table, we are at Now + t1, were the old behavior is still working and the new behavior is deployed as well.

Projects using build.commands on their configuration file and/or opting-in into beta addons from project's setting are getting the new behavior: the JS client is automatically injected on each page.

Related blog posts:

I think this answers most of your questions and gives you a way to play around with this. You can see a live example of the implementation from our theme in our own documentation (it uses the PR that I shared previously): https://docs.readthedocs.io/en/stable/

3. Could you help me figure out what would be equivalent to the things I'm doing in Furo's conf.py gated behind a dev-only conditional + some static files, which functionally is trying to inject all the pieces that RtD's current build-process injection would inject?

Yes, but I'm leaving now 😄 . I will come to this issue tomorrow's morning.

humitos commented 2 months ago

3. Could you help me figure out what would be equivalent to the things I'm doing in Furo's conf.py gated behind a dev-only conditional + some static files, which functionally is trying to inject all the pieces that RtD's current build-process injection would inject?

I opened https://github.com/pradyunsg/furo/pull/797 to talk more about this directly in the code, since I think it will be easier to know exactly what we are doing and where. Let me know if this is helpful as a first step.

I will come to this issue tomorrow's morning.

It wasn't tomorrow, but close enough 😄

humitos commented 1 month ago

I just want to check here if you were able to take a look at the PR that I opened that explains a little how to integrate the new Read the Docs Addons. I'm happy to help here with this work, but I would need to understand how do you run the tests so I can experiment with that. Let me know if there is anything else here I can do to help with this.

humitos commented 1 week ago

I wanted to share an update about our deprecation plan here ^1:

Soon, we will disable the Sphinx context injection to all new projects created on Read the Docs. That means that a bunch of variables that were available on Sphinx templates won't be available anymore.
New projects will have all Read the Docs Addons enabled by default.
We've created a Sphinx extension (https://github.com/readthedocs/sphinx-build-compatibility/) for users that want to opt-in for the old behavior --this extension will inject all those variables that we are removing. We will be recommending this extension as temporal solution while authors/themes migrate their code. However, we won't maintain this extension for too long.
After some time testing this new implementation, we will disable the Sphinx context injection on all projects and enable addons on all projects at the same time.

I'd appreciate any feedback you may have about this plan. Also, I'm happy to help you with any code modification required on furo or on any other theme you are involved. Let me know 👍🏼

humitos commented 5 days ago

🗞️ We are about to publish a post in our blog about this plan: https://github.com/readthedocs/website/pull/308

pradyunsg commented 5 days ago

This has been on my radar but free time hasn't been coming around often enough to get to this. Pesky things like cutting pip releases keep getting in the way.

pradyunsg / furo