OpenEugene / little-help-book-web

Human service resource guide powered by White Bird Clinic
MIT License
6 stars 4 forks source link

Use cross-platform friendly character encoding in get_table.py #168

Open coatsd opened 3 years ago

coatsd commented 3 years ago

image The current code in get_table.py uses a platform specific character encoding method that fails on other operating systems. Ideally, we would use a standardized encoding that would allow us to output it to a file on any operating system. This stackoverflow issue discusses this issue: https://stackoverflow.com/questions/27092833/unicodeencodeerror-charmap-codec-cant-encode-characters

colindavey commented 3 years ago

I discovered this problem when putting the python code into Docker. It worked fine on my computer, so I didn't know there was a problem until trying under Docker. I tried a number of fixes that didn't work, including adding encoding="utf-8" as suggested in the stackoverflow article.

One thing I discovered is that python 3.7 has a fix for this, and my computer was running 3.7, which is why never saw the problem. On the Docker side, what I did, in conjunction with MVP studio, was to upgrade the base Linux image to a newer one that has python 3.7.

Before that, I found that I could fix the problem in Docker with this environment variable: export LC_CTYPE="C.UTF-8". Oliver Dain suggested doing this: https://www.tecmint.com/set-system-locales-in-linux/, but we didn't pursue it after deciding to upgrade the Docker image.

colindavey commented 3 years ago

One decision we have is whether this code will continue to live in the lhb web repo, and be run by developers in their terminal, or whether it will be moved to the lhb py repo, where it will be primarily run as a web app. At this point, both repos have it.