search

jquast / wcwidth

Python library that measures the width of unicode strings rendered to a terminal
Other
391 stars 58 forks source link

Combining browser #5

Closed thomasballinger closed 9 years ago

thomasballinger commented 9 years ago

We've updated created a new script for testing whether combining characters combine as expected, and found that many don't on our terminals. Should we consider the behavior on our terminals correct, or read more about all the offending classes of characters?

Any feedback on the code would be great too :)

jquast commented 9 years ago

Absolutely, great work. Looks like we'll have to add a few unit tests to go with, too. One of them is failing for the old behavior.

I'll check how things appear in wcwidth-browser.py after work. Thanks again!

jquast commented 9 years ago

These have been merged and docs have been updated to reflect the behavior of non-zero-width combining characters and accredit the contributions. Thanks for taking the time!

The changes proposed in wcwidth-combining-browser.py has been integrated directly with wcwidth-browser.py, and various formatting (column alignment, etc.) issues have been resolved. To see the changes, make sure to run python setup.py develop once more before launching. Use the command-line option --combining to take a look.

I spent a while inspecting the discrepancies -- that some columns are not properly aligned. These are due to my terminal emulator (iTerm2, OSX) using the libc wcwidth(3) provided by the operating system, which is based on a much older unicode specification (probably Markus Kuhn's -- version 5.0).

If you pipe the output of wcwidth-browser.py to less(1) you will see that anything that renders (recognized by libc) will typically line up just fine. You may notice some are thick boxey blobs, this is because the font does not contain these codepoints and these are place-ins, but they otherwise align fine (albeit ugly).

(Also spent a while implementing static analysis code checks and adding well-formed docstrings to all of the files, which caused me to make a PR of my own https://github.com/landscapeio/prospector/pull/71)

thomasballinger commented 9 years ago

Thanks so much for the support and for finishing this off for us! @lmontopo let's take a look at this sometime next week.

lmontopo commented 9 years ago

Yes this was fun! Looking forward to chatting more with you next week @thomasballinger!