xfq
commented
1 day ago Some preliminary analysis:
Usage statistics of GB 18030, GBK, and GB 2312 for websites
See:
- https://w3techs.com/technologies/details/en-gb18030
- https://w3techs.com/technologies/details/en-gbk
- https://w3techs.com/technologies/details/en-gb2312
Bing search engine results
Searching [] (U+E5E5) in Bing China, most of the web pages were in 200x. Some websites may quote text from other websites, resulting in some recent results:
| Website | Results | Latest year in the first page | Bing International |
|---|---|---|---|
| sina.com.cn | 1,730,000 | 2014 | |
| Sohu.com | 778,000 | 2024 | |
| qq.com | 78,700 | 2021 | |
| Sogou.com | 67,900 | 2021 | |
| Zol.com.cn | 42,300 | 2024 | |
| Jjwxc.net | 39,000 | 2010 | |
| people.com.cn | 27 | 2022 | |
| chinanews.com.cn | 14 | ||
| Pconline.com.cn | 11 | 2024 | |
| china.com | 10 | 2024 | |
| jd.com | 7 | 2024 | |
| Hexun.com | 3 | 2003 | |
| alipay.com | 0 | 75,000 | |
Examples
https://www.chinanews.com.cn/n/2004-01-13/26/391173.html (GB 2312)
This website uses both U+3000 and U+E5E5.
https://blog.sina.com.cn/s/blog_44c67f2c0102v4at.html (UTF-8)
This website uses both U+3000 and U+E5E5.
https://news.sina.com.cn/c/2004-01-21/09272687351.shtml (GB 2312)
This website only uses U+3000.
https://edu.sina.com.cn/focus/wq3/index.html (GB2312, date 2008)
Although it's a U+E5E5 result, the source code only contains U+3000.
hsivonen
vyv03354
annevk
What is the issue with the Encoding Standard?
https://encoding.spec.whatwg.org/#gb18030-encoder
We didn't update this in https://github.com/whatwg/encoding/pull/336 , so I filed this issue to track it.
https://bugzilla.mozilla.org/show_bug.cgi?id=131837 , a bug filed in 2002 mentioned this. The reason behind this mapping was that some websites use 0xA3 0xA0 as space characters, which causes display abnormalities, so Mozilla changed the mapping to
U+3000 IDEOGRAPHIC SPACE.In the Hong Kong Supplementary Character Set, U+E5E5 was used to encode
𨪜(U+2A89Cin Unicode CJK Unified Ideographs Extension I).We need to analyze how many websites using GB 18030 are still using 0xA3 0xA0 to represent U+3000.
Currently, iconv
and ICUseems to map 0xA3 0xA0 to U+E5E5. ICU 74.1+ maps it to U+3000.The following is some information about this misuse (mostly translated from a Chinese website).
The 0xA3A1 ~ 0xA3FE part of GB18030-2022 is inherited from row 3 of GB 2312, and contains the G0 set of GB/T 1988-80 (ISO 646-CN). GB 2312 does not specify the width of these characters, but subsequent standards (such as GB 5007.1-85) made it clear that characters in row 3 are full-width, which are mapped to the Halfwidth and Fullwidth Forms Unicode block.
However, the G0 set of GB/T 1988-80 does not include spaces, but influenced by ASCII, people often consider spaces together with the remaining 94 characters. Now let's assume that someone thinks that 0xA3A1 ~ 0xA3FE are full-width ASCII characters (although "＄" has been replaced by "￥"), then this person is likely to think that 0xA3 0xA0 should be a full-width space (although the actual full-width space is at 0xA1A1). Because some fonts display .notdef as a 1 em wide space, even when the corresponding Unicode code point of the two are different, the rendering is the same (undefined PUA code points in GB encoding will be displayed as .notdef).