Support different cultures & languages

alchexmist commented 5 years ago

I would like to know if there is any possibility of adding support for "gettext".  I need to generate documentation for a project: English & Spanish & Galician.

A too complex solution: https://elixirforum.com/t/ex-doc-spanish-language-support/23140

Thanks in advance for any help you are able to provide.

josevalim commented 5 years ago

We would accept a pull request but note that only a minor part of the content is handled by ExDoc. Mostly section titles. Everything else comes from the project itself. So the work would be in writting your own documentation in multiple languages.

alchexmist commented 5 years ago

Ok. I will try update the templates myself.

Thanks for the quick response!

wojtekmach commented 5 years ago

@alchexmist, @joaquinalcerro is working on something similar so I'd suggest to talk to each other!

alchexmist commented 5 years ago

@alchexmist, @joaquinalcerro is working on something similar so I'd suggest to talk to each other!

Thank you for the suggestion. Unfortunately, not much time is left to reach the deadline for the project. I would need a quick solution.

Anyway, thank you so much.

joaquinalcerro commented 5 years ago

I managed to make some changes to the templates titles and subtiles using a labels configuration inside the project (mix.exs) configuration as follows:

defp docs do
    [
      main: "readme",
      labels: %{
        summary: "Resumen",
        sub_titles: %{
          callbacks: "Devoluciones de llamadas"
        },
        functions: "Funciones"
      },
      extras: [
        "README.md",
        "CHANGELOG.md"
      ],
.....
end

It worked fine but running the test failed so my next step was to review the code and fix it.

My changes where focused on the HTML Formatter so I am still missing the EPUB part.

You can check the changes here: https://github.com/joaquinalcerro/ex_doc

Am I on the right track or should I start over with a different approach.

Best regards,

alchexmist commented 5 years ago

Hi all,

I would adopt a ("standard") Gettext-based approach (i18n and l10n). I would separate by locale:

Package (required): ex_doc (international english).
Package (optional): ex_doc_locale_es_ES (spanish - spain).
Package (optional): ex_doc_locale_gl_ES (galician - spain).

http://blog.plataformatec.com.br/2016/03/using-gettext-to-internationalize-a-phoenix-application/

https://github.com/elixir-lang/gettext

joaquinalcerro commented 5 years ago

I still haven't worked with gettext. I will look into it.

Just to be clear about the requirements. Are you expecting to implement gettext on all the ex_doc text or just titles and sub titles like this:

Best regards,

alchexmist commented 5 years ago

Just to be clear about the requirements. Are you expecting to implement gettext on all the ex_doc text or just titles and sub titles like this:

As José Valim said:

only a minor part of the content is handled by ExDoc. Mostly section titles (@josevalim)

Just titles, subtitles and sidebar:

Search -> Buscar.
Guides -> Guías.
Exceptions -> Excepciones.
Mix Tasks -> Tareas de Mix.
API Reference -> Referencia de la API.

Best regards

cybrox commented 5 years ago

I would separate by locale: Package (required): ex_doc (international english). Package (optional): ex_doc_locale_es_ES (spanish - spain). Package (optional): ex_doc_locale_gl_ES (galician - spain).

I think creating a separate package for every locale is a bit of an overkill. Given that most people will only use ex_doc in english anyways, it would probably be easiest for people to just add e.g. ex_doc_translations when they want to use any other locale.

If you're looking for a very quick solution, forking ex_doc and just changing the template string is the way to go.

michalmuskala commented 5 years ago

A thing to consider is also that given how small the amount of things to translate actually is, it could be fine to bundle it all into ex_doc itself. It's just a dev-dependency anyway, so I think the size constraint is a bit less of an issue.

josevalim commented 5 years ago

Yes, a translations key or module may be enough.

cybrox commented 5 years ago

It's just a dev-dependency anyway, so I think the size constraint is a bit less of an issue.

That's a very good point.

Considering the way merge_config/1 works, the default english locales could stored in ExDoc.Config and could be overwritten by the user. That would essentially be what @joaquinalcerro has started implementing

whatyouhide commented 5 years ago

Whatever we end up doing, I think gettext would be nice to be able to handle the translations in PO files (which are very nice to work with) instead of source code.

alchexmist commented 5 years ago

I think it would be possible to re-use PO files from other free/open source projects (MIT, Apache2, etc.)

alchexmist commented 5 years ago

I do not have free time! This is my proposal (commit): https://github.com/alchexmist/ex_doc/commit/6c92f6eba0252667e7422c33d9d3fd86f8f73a69

Workflow (PO files): https://github.com/elixir-lang/gettext/blob/master/README.md

Things to fix:

Gettext.put_locale(ExDoc.Gettext, "es_ES") location.
how to set the second argument: es_ES, en_US, ...

With enough time it is very easy!

joaquinalcerro commented 5 years ago

@alchexmist,

Gettext looks pretty simple. Apparently this way you don't have to change much code.

I can put some time in this feature this weekend.

Can we all agree gettext is the way to go? Just to be sure I will be on the right track.

josevalim commented 5 years ago

I think gettext is a great tool but someone should investigate how someone will set the locale and load .po files in a project that is external to ex_doc. It totally works when changing ExDoc itself, but the API we are going for means that someone will use ex_doc as a dependency and then configure it.

alchexmist commented 5 years ago

Hi all,

@josevalim Would it be possible to use :language option? (mix help docs)

:language - Identify the primary language of the documents, its value must be a valid BCP 47 (https://tools.ietf.org/html/bcp47) language tag; default: "en"

   def project do
      [app: :my_app,
       version: "0.1.0-dev",
       deps: deps(),

       # Docs
       name: "My App",
       source_url: "https://github.com/USER/PROJECT",
       homepage_url: "http://YOUR_PROJECT_HOMEPAGE",
       docs: [main: "MyApp", # The main page in the docs
              logo: "path/to/logo.png",
              language: "fr-FR",
              extras: ["README.md"]]]
    end

josevalim commented 5 years ago

We could, but that's not the problem. The problem is making ExDoc compile its gettext backend aware of external .po files. Maybe you would have to pass an external backend as argument:

language: "fr-FR", getttext: SomeApp.Gettext

michalmuskala commented 5 years ago

I think ex_doc should just ship with ready translations and don't provide extensibility there.

whatyouhide commented 5 years ago

I don't think we should ship ex_doc with translations of headers and small stuff when we don't have a good support system for translating the actual documentation.

michalmuskala commented 5 years ago

On one side, I agree, on the other I don't think there's anything that says @doc is inherently English. I think the primary concern for now is with entire documentation in another language, not with documentation translated to multiple languages. The later is a much bigger endeavour.

joaquinalcerro commented 5 years ago

In my opinion, if someone needs documentation in another language than English, say Spanish, they should write the hole documentation in Spanish from scratch. "Automatically" translating technical documentation from any language to any language may introduce words that may change the meaning of what you cited in the other language and confuse the reader of the translated documentation.

The question is why translate the documentation (the use case): I think that a translation will be more useful and worth it when it will be used by a internal team of developers or you are going to share the documentation with other people that will use your application in a technical way (ex. API documentation for a private application) and they all speak the same language.

If someone wants to share a library to the world, maybe a open source project, they are going to have to write it in English because it is a universal language.

With that been said, I would focus on translating only titles and subtitles as the initial request and similar to what @alchexmist showed in his previous post.

I do not have free time! This is my proposal (commit): alchexmist@6c92f6e

Workflow (PO files): https://github.com/elixir-lang/gettext/blob/master/README.md

Things to fix:

Gettext.put_locale(ExDoc.Gettext, "es_ES") location.

how to set the second argument: es_ES, en_US, ...

With enough time it is very easy!

alchexmist commented 5 years ago

In my opinion, I think perspective is being lost on this issue.

I agree with @joaquinalcerro:

I think that a translation will be more useful and worth it when it will be used by a internal team of developers or you are going to share the documentation with other people that will use your application in a technical way (ex. API documentation for a private application) and they all speak the same language.

It would be convenient to highlight at least four ideas:

"Only a minor part of the content is handled by ExDoc". (@josevalim)
- translate([titles, subtitles, sidebar]) & Unicode === Full language support.
The translations would remain unchanged.
Translate the current documentation is not the goal.
"Make everything as simple as possible, but not simpler".

I don't think we should ship ex_doc with translations ... @whatyouhide

it would probably be easiest for people to just add e.g. ex_doc_translations when they want to use any other locale. @cybrox

ex_doc_translations package -> ex_doc/priv/gettext ??

josevalim commented 5 years ago

The boring part of this issue is not choosing which tool to use, but rather finding all strings and extracting then out. So my proposal is to find all strings in the source code we want to translate and move it a translate(config, "String") function. The default implementation of this function is:

def translate(_, text), do: text

Then please send a PR which we will merge. Once that is done, we will be able to better assess how complex the issue is and choose a solution. Implementing a solution will be easier once all uses have been identified.

alchexmist commented 5 years ago

@joaquinalcerro , I have sent the pull request: https://github.com/elixir-lang/ex_doc/pull/1030

joaquinalcerro commented 5 years ago

Oh... that was fast... sorry for not being fast enough... glad to help any time. Best regards.

elixir-lang / ex_doc

Support different cultures & languages #1027