5c59cbd7-8186-4351-8391-b403f3a3a73f
commented
13 years ago Unicode character names share a common namespace with formal aliases and with named sequences, but Python recognizes only the original name. That means not everything in the namespace is accessible from Python. (If this is construed to be an extant bug from than an absent feature, you probably want to change this from a wish to a bug in the ticket.)
This is a problem because aliases correct errors in the original names, and are the preferred versions. For example, ISO screwed up when they called U+01A2 LATIN CAPITAL LETTER OI. It is actually LATIN CAPITAL LETTER GHA according to the file NameAliases.txt in the Unicode Character Database. However, Python blows up when you try to use this:
% env PYTHONIOENCODING=utf8 python3.2-narrow -c 'print("\N{LATIN CAPITAL LETTER OI}")'
Ƣ
% env PYTHONIOENCODING=utf8 python3.2-narrow -c 'print("\N{LATIN CAPITAL LETTER GHA}")'
File "<string>", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-27: unknown Unicode character name
Exit 1This unfortunate, because the formal aliases correct egregious blunders, such as the Standard reading "BRAKCET" instead of "BRACKET":
$ uninames '^\s+%'
Ƣ 01A2 LATIN CAPITAL LETTER OI
% LATIN CAPITAL LETTER GHA
ƣ 01A3 LATIN SMALL LETTER OI
% LATIN SMALL LETTER GHA
* Pan-Turkic Latin alphabets
ೞ 0CDE KANNADA LETTER FA
% KANNADA LETTER LLLA
* obsolete historic letter
* name is a mistake for LLLA
ຝ 0E9D LAO LETTER FO TAM
% LAO LETTER FO FON
= fo fa
* name is a mistake for fo sung
ຟ 0E9F LAO LETTER FO SUNG
% LAO LETTER FO FAY
* name is a mistake for fo tam
ຣ 0EA3 LAO LETTER LO LING
% LAO LETTER RO
= ro rot
* name is a mistake, lo ling is the mnemonic for 0EA5
ລ 0EA5 LAO LETTER LO LOOT
% LAO LETTER LO
= lo ling
* name is a mistake, lo loot is the mnemonic for 0EA3
࿐ 0FD0 TIBETAN MARK BSKA- SHOG GI MGO RGYAN
% TIBETAN MARK BKA- SHOG GI MGO RGYAN
* used in Bhutan
ꀕ A015 YI SYLLABLE WU
% YI SYLLABLE ITERATION MARK
* name is a misnomer
︘ FE18 PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRAKCET
% PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRACKET
* misspelling of "BRACKET" in character name is a known defect
# <vertical> 3017
𝃅 1D0C5 BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS
% BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS
* misspelling of "FTHORA" in character name is a known defectThere are only
In Perl, \N{...} grants access to the single, shared, common namespace of Unicode character names, formal aliases, and named sequences without distinction:
% env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER OI}")'
Ƣ
% env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER GHA}")'
Ƣ
% env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER OI}")' | uniquote -x
\x{1A2}
% env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER GHA}")' | uniquote -x
\x{1A2}It is my suggestion that Python do the same thing. There are currently only 11 of these.
The third element in this shared namespace of name, named sequences, are multiple code points masquerading under one name. They come from the NamedSequences.txt file in the Unicode Character Database. An example entry is:
LATIN CAPITAL LETTER A WITH MACRON AND GRAVE;0100 0300There are 418 of these named sequences as of Unicode 6.0.0. This shows that Perl can also access named sequences:
$ env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}")'
Ā̀
$ env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}")' | uniquote -x
\x{100}\x{300}
$ env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{KATAKANA LETTER AINU P}")'
ㇷ゚
$ env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{KATAKANA LETTER AINU P}")' | uniquote -x
\x{31F7}\x{309A}Since it is a single namespace, it makes sense that all members of that namespace should be accessible using \N{...} as a sort of equal-opportunity accessor mechanism, and it does not make sense that they not be.
Just makes sure you take only the approved named sequences from the NamedSequences.txt file. It would be unwise to give users access to the provisional sequences located in a neighboring file I shall not name :) because those are not guaranteed never to be withdrawn the way the others are, and so you would risk introducing an incompatibility.
If you look at the ICU UCharacter class, you can see that they provide a more
terryjreedy
39d85a87-36ea-41b2-b2bb-2be43abb500e
gvanrossum
ezio-melotti
61337411-43fc-4a9c-b8d5-4060aede66d0
1762cc99-3127-4a62-9baf-30c3d0f51ef7
abalkin
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields: ```python assignee = 'https://github.com/ezio-melotti' closed_at =
created_at =
labels = ['interpreter-core', 'type-feature', 'expert-unicode']
title = '\\N{...} neglects formal aliases and named sequences from Unicode charnames namespace'
updated_at =
user = 'https://bugs.python.org/tchrist'
```
bugs.python.org fields:
```python
activity =
actor = 'belopolsky'
assignee = 'ezio.melotti'
closed = True
closed_date =
closer = 'ezio.melotti'
components = ['Interpreter Core', 'Unicode']
creation =
creator = 'tchrist'
dependencies = []
files = ['22903', '23273', '23280', '23291', '23374']
hgrepos = []
issue_num = 12753
keywords = ['patch', 'needs review']
message_count = 37.0
messages = ['142136', '142145', '142502', '142506', '142507', '142508', '143043', '144679', '144681', '144703', '144708', '144716', '144738', '144739', '144757', '144758', '144760', '144779', '144783', '144802', '144803', '144825', '144827', '144832', '144836', '144839', '145254', '145263', '145327', '145401', '146034', '146036', '146075', '146114', '146129', '146135', '191737']
nosy_count = 10.0
nosy_names = ['lemburg', 'gvanrossum', 'loewis', 'terry.reedy', 'belopolsky', 'ezio.melotti', 'mrabarnett', 'flox', 'python-dev', 'tchrist']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = '4610'
type = 'enhancement'
url = 'https://bugs.python.org/issue12753'
versions = ['Python 3.3']
```