search

sphinx-doc / sphinx

The Sphinx documentation generator
https://www.sphinx-doc.org/
Other
6.56k stars 2.12k forks source link

Gettext does not add substitutions to message catalogue #4293

Open TheRealMoeder opened 6 years ago

TheRealMoeder commented 6 years ago

Problem

Running gettext doesn't get messages within substitutions.

Procedure to reproduce the problem

sphinx-build -b gettext

example conf.py:

source_suffix = '.rst'
master_doc = 'index'
gettext_compact = False

index.rst:

Line 1

.. |testsub| replace:: test-substitution

Bla bla bla.

Testing |testsub|

Environment info

tk0miya commented 6 years ago

@shimizukawa could you check this please? AFAIK, this behavior is a specification of gettext builder, right?

TheRealMoeder commented 6 years ago

Any update on this issue or a suggested workaround?

tk0miya commented 6 years ago

As I commented above, this might be a specification of gettext builder. I think there are no workaround for it.

shimizukawa commented 6 years ago

AFAIK, this behavior is a specification of gettext builder, right?

Before #4840 (merged at 2018/4/14), Yes, gettext builder (more accurately, on sphinx/util/nodes.py) makes invisible nodes (including substitution_definition node) out of the scope of i18n. https://github.com/sphinx-doc/sphinx/blob/feee462836677b23041a910734be221bcc0731bb/sphinx/util/nodes.py#L102-L103

Since #4840, gettext builder can not grab substitution_definition node anymore because the node has been removed by the SubstitutionDefinitionsRemover transform on reading phase. https://github.com/sphinx-doc/sphinx/blob/feee462836677b23041a910734be221bcc0731bb/sphinx/io.py#L99

For this reason, substitution text can not be internationalized with a little modification.

@tk0miya because substitution_definition node has been removed on reading phase, an option to generate gettext message from document.substitution_defs is also considered to generate message with gettext builder, but I think it is not a best way. How do you think?

tk0miya commented 6 years ago

Surely, it would be better to make SubstitutionDefinitionsRemover as a post transform.

sfc-gh-arozanski commented 2 months ago

Hey @shimizukawa, Since https://github.com/sphinx-doc/sphinx/pull/8354 has been merged, it should now be possible to access the substitution_definition node during the string extraction phase, right?

Is there something else blocking this task? I can try to contribute a PR if you give me some hints on how to tackle that.

shimizukawa commented 2 months ago

@sfc-gh-arozanski Thank you for your suggestion. I'll tell you what I've found out so far. It's been a while, so it may not be accurate. If you search the entire source code for substitution_definition, you'll get 13 hits. Of these, gettext.py , test_build_gettext.py , and test_intl.py seem to be related.

/workspaces/sphinx (master) $ grep -r substitution_definition --exclude-dir "*pycache*" *
doc/changes/1.8.rst:* #4827: All ``substitution_definition`` nodes are removed from doctree on
sphinx/builders/gettext.py:            if not _is_node_in_substitution_definition(node):
sphinx/builders/gettext.py:def _is_node_in_substitution_definition(node: nodes.Node) -> bool:
sphinx/builders/gettext.py:        if isinstance(node, nodes.substitution_definition):
sphinx/builders/latex/transforms.py:    """Remove ``substitution_definition`` nodes from doctrees."""
sphinx/builders/latex/transforms.py:        for node in list(self.document.findall(nodes.substitution_definition)):
sphinx/writers/texinfo.py:    def visit_substitution_definition(self, node: Element) -> None:
sphinx/writers/text.py:    def visit_substitution_definition(self, node: Element) -> None:
tests/test_intl/test_intl.py:    testroot='intl_substitution_definitions',
tests/test_intl/test_intl.py:def test_additional_targets_should_be_translated_substitution_definitions(app):
tests/test_intl/test_intl.py:    testroot='intl_substitution_definitions',
tests/test_builders/test_build_gettext.py:    testroot='intl_substitution_definitions',
tests/test_builders/test_build_gettext.py:    testroot='intl_substitution_definitions',

From test_intl.py you can see that the directory tests/roots/test-intl_substitution_definitions/ is used for testing content.

Also, the IGNORE_NODES I mentioned in this issue is in sphinx/util/nodes.py. I think the is_translatable function in this file excludes substitutions. Please check by outputting the log.