gettext translation within component inline templates is not working

[nerdoc]

It seems that Django's makemessages does not recognize/find a simple translated string within a component's inline template:

@default.register
class Foo(Component):
    template: django_html = """{% load i18n %}<div>{% translate "foo" %}</div>"""

When using template_name and a file, it is handled correctly, as expected.

I don't know exactly where this must be handled. InlineTemplate? @samwillis - any ideas?

[nerdoc]

The only solution ATM is to use template_name instead of template and use a .html file which Django can parse during manage.py makemessages.

Overriding / patching Django's behaviour here seems to be a bit overwhelming. Evtl. tetra could add a translate "proxy" templatetag that writes the string into a separate .py file using gettext_noop, so that at least the strings get extracted from there during the next makemessages run.

But that makes me shiver a bit if I think about it...

[nerdoc]

@samwillis So you have any idea how to get into this? Just need a hint here. gettext and Django makemessages do not support "plugins" or flexible parsing. This smells like an ugly hack.

[samwillis]

Hey @nerdoc

it looks like Django explicitly doesn't use the "django" domain for .py files, which is understandable as it expecting normal gettext behaviour in a .py:

https://github.com/django/django/blob/1a36dce9c5e0627d46582829d7abd47ed872e3aa/django/core/management/commands/makemessages.py#L87

I would look at overriding the makemessages command like tetra does for startserver with an option to extract messages from templates in .py files. https://docs.djangoproject.com/en/5.0/topics/i18n/translation/#customizing-the-makemessages-command

[nerdoc]

That's a good catch. I'll look into that, thanks!

[nerdoc]

Tetra could overwrite the is_templatized method and "parse" the file (I asked ChatGPT to generate some code here):

    def is_templatized(self):
        if self.domain == "djangojs":
            return self.command.gettext_version < (0, 18, 3)
        elif self.domain == "django":
            file_ext = os.path.splitext(self.translatable.file)[1]
            if file_ext == ".py":
                with open(self.translatable.file, "r") as file:
                    content = file.read()
                    try:
                        tree = ast.parse(content)
                        for node in ast.walk(tree):
                            if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == "Component":
                                return True
                    except SyntaxError:
                        return False
            return file_ext != ".py"
        return False

This would be horribly inefficient and slow when firing makemessages, but could be a starting point.

[samwillis]

this is a good plan.

You can probably infer if there might be a component by checking if the files text includes the correct import statements, and only parse it if so. That would make it significantly more efficient.

[nerdoc]

There may be multiple ways of importing Tetra components.

from tetra.components import Component  # -> class Foo(Component)
from tetra import components  # -> class Foo(components.Component)
from tetra.components import component as Baz  # -> class Foo(Baz)
from my_app import FooComponentBase  # -> class Foo(FooComponentBase)

Which are all correct imports, but makes parsing more complicated... I'll check how this impacts performance. makemessages is not a performance problem when run from time to time...

[nerdoc]

class BuildFile(MakeMessagesBuildFile):
    def is_templatized(self):
        if self.domain == "django":
            file_ext = os.path.splitext(self.translatable.file)[1]
            if file_ext == ".py":
                with open(self.translatable.file, "r") as file:
                    content = file.read()
                    if "from tetra.components import Component" in content:
                        try:
                            tree = ast.parse(content)
                            for node in ast.walk(tree):
                                if isinstance(node, ast.ClassDef):
                                    has_component_base = False
                                    has_template_attr = False
                                    for base in node.bases:
                                        if (
                                            isinstance(base, ast.Attribute)
                                            and base.attr == "Component"
                                            and isinstance(base.value, ast.Name)
                                            and base.value.id == "tetra.components"
                                        ):
                                            has_component_base = True
                                    for stmt in node.body:
                                        if isinstance(stmt, ast.Assign):
                                            for target in stmt.targets:
                                                if (
                                                    isinstance(target, ast.Name)
                                                    and target.id == "template"
                                                ):
                                                    has_template_attr = True
                                    if has_component_base and has_template_attr:
                                        return True

                        except SyntaxError:
                            return False
                return file_ext != ".py"
            return super().is_templatized()

smashed together by ChatGPT. Leave it here as lift-off point, have no time this evening for real coding :-(

[nerdoc]

This is a major issue and not easy to solve. makemessages has a multi-staged process of parsing files and preparing them for gettext. I subclassed BuildFile and tried to change the way it preprocess the files. But this does not really work, as in fact Django uses django.utils.translation.templatize(src, origin=None) to prepare files for gettext. What would need to be done is hook into that process before the file is preprocessed, see if there is a Tetra component, and "extract" the template string from the component, writing a separate tmp file with it's content. This tmp file then must be added to the files_list and hence fed to the gettext program. The line numbers must be matched to the original document. So subclassing the BuildFile is too late in the process, as this only encapsulates ONE file.

This seems such a complicated task that it occurs to me that there must be something wrong.

My first mental approach would be: why this complicated? Why don't we support (or even recommend) building a component in a directory instead of a single file?

my_component/
  __init__.py     # or my_component.py
  my_component.js   
  my_component.css 
  my_component.html

Then all those problems would be solved, including caching, IDE/highlighting support etc. This is what other component frameworks do as well.

But @samwillis - I think you mad this one-file approach with a clear goal in mind. The problem here is that, without huge effort, translating is not possible in a component - which is VeryBad™.

If anyone has an idea, please elaborate!

[nerdoc]

Ah, it's always the same. Countless hours of coding, reading, thinking. Then, I decide to ask in a forum, open an issue, go for help. And a few minutes later a new idea comes up, and that path solves the issue.

I seem to have done it. It's (as always) easier than initially thought: The TetraBuildFile class must check if a .py file is "templatized". It does this by parsing the code and checking if this file contains a component. And when it is, it just returns True. Everything else is done by Django.

The only issue remaining is that the code line referenced in the .po file is the line the inline template starts. This could be done in another step. But at least, this is solved. Fix as commit later today.