Unicode NFD is destructive

rurban / ctl

My variant of the C Template Library

https://rurban.github.io/ctl/

MIT License

172 stars 8 forks source link

Unicode NFD is destructive #23

Open TinoDidriksen opened 2 years ago

TinoDidriksen commented 2 years ago

u8string will get proper utf-8/unicode support, exceeding C++ STL. compare will check u8strings normalized to NFD.

Please consider that very carefully. Normalization is destructive. E.g. both Å (U+00C5) and Å (U+212B) will normalize to Å (U+0041 U+030A). There are lots of such destructive normalizations, but do you really want to consider them equal? I sure don't.

rurban commented 1 year ago

Compare is done with a copy of course. See also libu8ident, which does the ident checks already