apparently there are some problems with how the entity JSON used. There are conversions that consists of multiple codepoints, the notorious example being =⃥ here - (61 8421). It's simply generated as the first codepoint, that is, the equal sign, therefore equal signs can (and will) turn into =⃥ rather than correctly to =.
To avoid this, it would be good to use a list of codepoints rather than just one, or possibly use the grapheme-level representation on NQP level.
lizmat
commented
1 month ago
Hello,
apparently there are some problems with how the entity JSON used. There are conversions that consists of multiple codepoints, the notorious example being
=⃥here -(61 8421). It's simply generated as the first codepoint, that is, the equal sign, therefore equal signs can (and will) turn into=⃥rather than correctly to=.To avoid this, it would be good to use a list of codepoints rather than just one, or possibly use the grapheme-level representation on NQP level.