python-babel / babel

The official repository for Babel, the Python Internationalization Library
http://babel.pocoo.org/
BSD 3-Clause "New" or "Revised" License
1.29k stars 433 forks source link

Add f-string parsing for Python 3.12 (PEP 701) #1027

Closed encukou closed 9 months ago

encukou commented 9 months ago

Since Python 3.12, f-strings are tokenized and parsed like the rest of Python's grammar, using the new tokens FSTRING_START, FSTRING_MIDDLE and FSTRING_END, as described in pep 701 #new-tokens.

Make the babel message extractor concatenate these three if they're adjacent to each other. If they're not, that means there are dynamic substitutions, so the f-string is ignored.

encukou commented 9 months ago

Not sure if y'all want to add 3.12 to the CI yet? For that to pass setuptools will need to be added to tox.ini as it's needed in the data setup.

mgorny commented 9 months ago

Thanks. I can confirm that with this patch, the tests pass both on 3.11 and 3.12.

codecov[bot] commented 9 months ago

Codecov Report

Merging #1027 (8260a4d) into master (6fc07a2) will decrease coverage by 0.17%. Report is 1 commits behind head on master. The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master    #1027      +/-   ##
==========================================
- Coverage   90.94%   90.78%   -0.17%     
==========================================
  Files          25       25              
  Lines        4375     4393      +18     
==========================================
+ Hits         3979     3988       +9     
- Misses        396      405       +9     
Files Coverage Δ
babel/messages/extract.py 92.89% <50.00%> (-2.42%) :arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

akx commented 9 months ago

Not sure if y'all want to add 3.12 to the CI yet? For that to pass setuptools will need to be added to tox.ini as it's needed in the data setup.

Working on that in #1028 :)

BrentHuang commented 2 months ago

how to use? In babel 2.14.0, the code can't be extracted to .pot file:

foo= 'Hardy'
print(_(f"Hello {foo}"))
BrentHuang commented 2 months ago

how to use? In babel 2.14.0, the code can't be extracted to .pot file:

foo= 'Hardy'
print(_(f"Hello {foo}"))
foo= 'Hardy'
print(f'{_("Hello")} {foo}')

in the code above, extract ok, but not translated by gettext.