search

wichert / lingua

Translation toolkit for Python
Other
46 stars 32 forks source link

Crash in Python extractor when using a custom translation function #57

Closed ldaverio closed 9 years ago

ldaverio commented 9 years ago

After upgrading from Lingua 1.6 to 3.9, and trying pot-create with an existing project, I came across a bug in the Python extractor. Some of the files of the projects had been processed by Modernize (https://pypi.python.org/pypi/modernize), and Unicode strings had been rewritten as six.u("...").

If a custom translation function is used on such "strings" (e.g. _other(...) instead of _(...)), the extractor will choke with the following error message:

Traceback (most recent call last):
  File "/.../bin/pot-create", line 11, in <module>
    sys.exit(main())
  File "/.../lib/python2.7/site-packages/lingua/extract.py", line 262, in main
    for message in extractor(real_filename, options):
  File ".../lib/python2.7/site-packages/lingua/extractors/python.py", line 90, in _extract_python
    msg = parse_keyword(node, KEYWORDS[node.func.id])
  File ".../lib/python2.7/site-packages/lingua/extractors/python.py", line 31, in parse_keyword
    msgid = node.args[keyword.msgid_param - 1].s
AttributeError: 'Call' object has no attribute 's'

The error can be reproduced by running pot-create --keyword=_other x.py on the following Python code. It will correctly handle lines 1-3, but choke on line 4:

_(u"This is extracted")
_(foo("This is ignored"))
_other(u"This is extracted, too")
_other(foo("Bug!"))
wichert commented 9 years ago

Lingua should not abort like that, but I don't see how it could ever handling something like _other(foo("Bug!")): there is no way for lingua to know what foo() will do and what it returns. The best I can do is fix the parser to not abort like this, but it will never do what you might want here.

Considering that Python 3.3 and later support u"..." I would strongly suggest to just use that and not use six.u(..) or an alternative for this kind of usage.

ldaverio commented 9 years ago

Hi :) My post was not about the extractor ignoring embedded function calls, which is normal, but specifically about crashing on _other(foo("Bug!")).

I'm not too familiar with Python 3 yet, so I figured that using Modernize was a good idea. But now I can see it isn't, and I'm going to follow your advice. Thank you!

PS: Actually, with Lingua 1.6, I had modified the extractor to take six.u() into account inside translator functions. It was an ugly hack, and I'm glad I can get rid of it :)