Allow for sematic url for i18n

ghassanmas commented 2 years ago

The Problem:

Right now the doc default url is https://docs.openedx.org/en/latest/* this would default en lang. However when changing the langauge using transifex selector lang it would change the cotent to the selected langauge, however the url would always contain en. This has the following downside:

The url is not semantic and is incorrect (in case user picked a different lang than en)
There isn't a mechanism to generate links for different lang code to force the docs site show content in a spesfic language.

Suggested Solutions

The good news is TX could view translated content if code exits in url; as per doc it can detect lang code in path i.e. /ar/ or in query params i.e ?lang=ar. However for this to work, from TX side we need enable lang detect which is a simple configuration can be added in this context : https://github.com/openedx/docs.openedx.org/blob/1e4c1adf38fd49ad99cba7582cfdbd14f25d764c/source/_templates/layout.html#L4-L7 We would have something like:

<script type="text/javascript">
    window.liveSettings = {
        api_key: "7af91b0a80054fab9de5fad172a25171",
         detectlang: true // This option can also be a function which should return the lang-code refer to refs below
    };

From readthedocs prespective (I am speculating here), to allow such featuer integration to work as expected, we would probably need to choose from the following options:

Remove /en/ part of the url (so then if we want to link a spesfic lang of the doc: we would just add ?lang=lang_code in the end of the url.
Allow readthedocs for whatever lang code exits in url to just serve the default page. i.e.
- For url https://docs.openedx.org/en/latest/* TX JS wouldn't do anything
- For url https://docs.openedx.org/ar/latest/*TX JS would mutate the DOM and show arabic version.

For second option above; since TX works in JS level (it mutate the dom content), it doesn't matter if Readthedocs would always serve english version for whatever code exits in the url. The caveate for this apporach is SEO, however SEO bots/indexer do run JS code , but of course to ensure we get the right langauge is being index we could consider prerender the site in different langauges (see ref 3), but then that might require more complex readthedocs configuration/integration. Although I don't know how important SEO is for doc site, I can nonetheless recommand if SEO is imporant for whichever approach we choose, someone would need to register the domain docs.openedx.org in Google search console so that we are confident content per langague are correrctly indexed.

How would this relate to bigger project:

Once this is resolved, in the context where the platform direct to extenral documenation source, we could direct learner/course author to the correct version of the doc that matches their language in account settings. Instead to just have hardcoded link to English.

I think this topic was once raised in TWG, to ensure the platform offers a consistent i18n experience. One blocker at that time was that docs is only available in en, but since its available in multiple lang and assuming this issue would be resolved. Then the last requirment to allow such expirence should be trivial or at least. We would need to ensure first for whatever external help url used, it should have a page/resource exits in this project and developer would be asked to add lang_code in the url.

Refs:

ehuthmacher commented 2 years ago

@feanil What are your thoughts regarding the two proposals? The SEO caveat is interesting. However, it will require a more extensive configuration in readthedocs.io.

feanil commented 1 year ago

@ghassanmas thanks for submitting this extermely thoughtful issue! I took a look through the translation documentation for ReadTheDocs. Based on that, I think it would be possible to implement the second option you suggested for any languages we want to support on the docs site. To do so, I think we'd have to do the following:

Enable language detection for transifex.
Setup docs.openedx.org in Google Search Console to be able to verify that it's doing the right thing for SEO and alternate language searchability.

Then for each Language we want to support, we would:

Create a new language specific project in RTD set to that language.
Setup that language to just compile the english text.
Link that new project to the existing docs site.

How many languages would we expect to turn on to begin with?

This task should also include writing any docs needed to be able to do this again for new languages in the future.

ghassanmas commented 1 year ago

@feanil that sounds reasonable, as per the requirments you listed above, here is my answer for them one by one:

Enable language detection for transifex.

I have opened a PR #167 for this. Note: this doesn't need to be repated when enabling/adding new lang...

Setup docs.openedx.org in Google Search Console to be able to verify that it's doing the right thing for SEO and alternate language searchability.

This I guess need to be done by someone who has access to DNS of docs.openedx.org, usually Google asks to add DNS record or serve spesfic file to verifiy that the person owns the domain/subdomain.

Then for each Language we want to support, we would:

Create a new language specific project in RTD set to that language. Setup that language to just compile the english text. Link that new project to the existing docs site.

That looks like what exacly we need, and I assume this would be done through RTD admin portal?

How many languages would we expect to turn on to begin with?

This I think best answerd by @ehuthmacher, to my knowledge we initally added support for langaues of which there are active participants for in TWG so that we test and get their feedback... So langagues are: es_419, fr_CA, ar, de_DE, it_IT, pt_PT. I got them from the TX project for docs.openedx.org without TX

This task should also include writing any docs needed to be able to do this again for new languages in the future.

Initally I would say if we stick to the current plan, then the steps to add new a lang is roughly:

From add new lang for the docs project in TX.
- Ensure X% of of content is reviewed/translated?
For RTD replicate the steps you mentiond above Probably we would end up with concerete list once we test this apporach with the one or more of the already enabaled langs..

Lastly I have other question, regarding the lang picker, right now we use TX built-in language picker, but I guess we might need to pick weather to stick to it (and if we stick to it, then we would want TX to change url once the lang is change see ref ^1) Or we disable TX lang picker and instaed rely on RTD.

feanil commented 1 year ago

Lastly I have other question, regarding the lang picker, right now we use TX built-in language picker, but I guess we might need to pick weather to stick to it (and if we stick to it, then we would want TX to change url once the lang is change see ref 1) Or we disable TX lang picker and instaed rely on RTD.

I think it would be better to stick to the RTD language picker and disable the TX built-in language picker since then it will more naturally look and work like other RTD sites. Having both seems like the wrong thing and I don't think there is a way to disable the RTD one if we enable multiple languages via RTD.

ghassanmas commented 1 year ago

I think it would be better to stick to the RTD language picker and disable the TX built-in language picker since then it will more naturally look and work like other RTD sites. Having both seems like the wrong thing and I don't think there is a way to disable the RTD one if we enable multiple languages via RTD.

Yes make sense, and that can be changed from TX project settings, no need to do anything code-wise

openedx / docs.openedx.org