Open tonywu6 opened 8 months ago
Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:
@tonywu6 I have added a PR #600 which should fix this. It would be great if you could try it out and ensure it works for you too.
@OriolAbril Can confirm that i18n is working as expected with this patch (tried on some of our repos)
Describe the bug
Sphinx uses gettext for internationalization: https://www.sphinx-doc.org/en/master/usage/advanced/intl.html
With this workflow, when translations are provided for notebooks, MyST-NB fails with a
nbformat.reader.NotJSONError
:Full traceback
``` Traceback (most recent call last): File ".venv/lib/python3.11/site-packages/nbformat/reader.py", line 20, in parse_json nb_dict = json.loads(s, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "~/.rye/py/cpython@3.11.8/install/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "~/.rye/py/cpython@3.11.8/install/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "~/.rye/py/cpython@3.11.8/install/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) The above exception was the direct cause of the following exception: Traceback (most recent call last): File ".venv/lib/python3.11/site-packages/sphinx/cmd/build.py", line 298, in build_main app.build(args.force_all, args.filenames) File ".venv/lib/python3.11/site-packages/sphinx/application.py", line 355, in build self.builder.build_update() File ".venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 293, in build_update self.build(to_build, File ".venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 313, in build updated_docnames = set(self.read()) ^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 420, in read self._read_serial(docnames) File ".venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 441, in _read_serial self.read_doc(docname) File ".venv/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 498, in read_doc publisher.publish() File ".venv/lib/python3.11/site-packages/docutils/core.py", line 236, in publish self.apply_transforms() File ".venv/lib/python3.11/site-packages/docutils/core.py", line 216, in apply_transforms self.document.transformer.apply_transforms() File ".venv/lib/python3.11/site-packages/sphinx/transforms/__init__.py", line 83, in apply_transforms super().apply_transforms() File ".venv/lib/python3.11/site-packages/docutils/transforms/__init__.py", line 182, in apply_transforms transform.apply(**kwargs) File ".venv/lib/python3.11/site-packages/sphinx/transforms/i18n.py", line 397, in apply patch = publish_msgstr(self.app, msgstr, source, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/sphinx/transforms/i18n.py", line 73, in publish_msgstr doc = reader.read( ^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/docutils/readers/__init__.py", line 70, in read self.parse() File ".venv/lib/python3.11/site-packages/docutils/readers/__init__.py", line 76, in parse self.parser.parse(self.input, document) File ".venv/lib/python3.11/site-packages/myst_nb/sphinx_.py", line 89, in parse notebook = nb_reader.read(inputstring) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/myst_nb/core/read.py", line 38, in standard_nb_read return nbf.reads(text, as_version=NOTEBOOK_VERSION) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/nbformat/__init__.py", line 89, in reads nb = reader.reads(s, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/nbformat/reader.py", line 76, in reads nb_dict = parse_json(s, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File ".venv/lib/python3.11/site-packages/nbformat/reader.py", line 26, in parse_json raise NotJSONError(message) from e nbformat.reader.NotJSONError: Notebook does not appear to be JSON: 'Franz jagt im komplett verwahrlosten Ta... Exception occurred: File ".venv/lib/python3.11/site-packages/nbformat/reader.py", line 26, in parse_json raise NotJSONError(message) from e nbformat.reader.NotJSONError: Notebook does not appear to be JSON: 'Franz jagt im komplett verwahrlosten Ta... ```Translations look like this
Sphinx will attempt to parse each
msgstr
using the parser configured for the corresponding source file (in this case the Notebook parser) because translations might contain markups:https://github.com/sphinx-doc/sphinx/blob/ae51974e217cc7590e1b9c68f18af3e043027c6d/sphinx/transforms/i18n.py#L72-L78
Perhaps fall back to MyST-Parser in this case?
Reproduce the bug
https://github.com/tonywu6/issues-repro-myst-nb-intl
List your environment