Closed ChiahongHong closed 2 months ago
Thank you!
@lukhnos, I noticed that we use utf8
in Source/Data/BPMFBase.txt
, would you help update the wiki to mention when to use utf8
and big5
respectively? Thanks!
@xatier @lukhnos
The following characters in BPMFBase.txt
are not included in Big5 but are marked as big5
, including extensions such as CP950 and HKSCS.
It seems that this field might have been deprecated?
Char | CP950 | HKSCS |
---|---|---|
〇 | Yes | |
亖 | ||
兲 | ||
叁 | Yes | |
叄 | ||
咔 | Yes | |
喆 | Yes | |
坂 | Yes | |
坔 | Yes | |
墻 | Yes | Yes |
夶 | ||
嫺 | Yes | Yes |
寗 | Yes | |
尛 | ||
弌 | Yes | |
弍 | Yes | |
弎 | Yes | |
彞 | ||
恒 | Yes | Yes |
晧 | Yes | |
温 | Yes | |
焿 | ||
犇 | Yes | |
着 | Yes | |
砈 | Yes | |
碁 | Yes | Yes |
礴 | Yes | |
竈 | Yes | |
粧 | Yes | Yes |
絝 | Yes | |
綉 | Yes | |
裏 | Yes | Yes |
酶 | Yes | |
醩 | Yes | |
銹 | Yes | Yes |
鍅 | Yes | |
闘 | ||
鬦 | ||
鬪 | Yes | |
鬬 | ||
鬭 | Yes | |
魩 | Yes | |
鮟 | Yes | |
鱇 | ||
鴴 | Yes | |
麪 | Yes | |
黒 | ||
𡻈 | Yes |
BTW, there's also an encoding cns
(CNS11643) that's not described in the Wiki:
是否是 Big5 編碼包含的字元,如果不是,就會標示為 UTF-8
cat BPMFBase.txt | grep cns
錱 ㄓㄣ zhen 5p cns
蟎 ㄇㄢˇ man3 a03 cns
躼 ㄔㄤˊ chang2 t;6 cns
畑 ㄊㄧㄢˊ tian2 wu06 cns
鱲 ㄌㄚˋ la4 x84 cns
栃 ㄌㄧˋ li4 xu4 cns
BPMFBase = set()
with open('BPMFBase.txt', 'r', encoding='UTF-8') as f:
for line in f.readlines():
char, _, _, _, encoding = line.split()
if encoding == 'big5':
BPMFBase.add(char)
print('Char\tCP950\tHKSCS')
print('═════════════════════')
for char in sorted(BPMFBase):
try:
char.encode('big5')
continue
except UnicodeEncodeError:
print(char, end='\t')
try:
char.encode('cp950')
print('Yes', end='\t')
except UnicodeEncodeError:
print(end='\t')
try:
char.encode('big5-hkscs')
print('Yes')
except UnicodeEncodeError:
print()
@ChiahongHong thanks for delving into this! This table is certainly helpful.
We should definitely update the wiki if these are not really used and perhaps run a script to remove this field.
此 PR 加入國教院所發布的臺灣華語文能力基準 (Taiwan Benchmarks for the Chinese Language, TBCL) 漢字表中所缺的字,只是剛好全部都是異體字。