internetarchive / warcprox

WARC writing MITM HTTP/S proxy
371 stars 54 forks source link

Avoid `UnicodeDecodeError` in Python 3.5, 3.6 #156

Closed ldko closed 3 years ago

ldko commented 3 years ago

Trying to install warcprox in Python 3.5 or 3.6 (using pip or setup.py) I was hitting this error:

  File "setup.py", line 51, in <module>
    long_description=open('README.rst').read(),
  File "~/.virtualenvs/warcprox_36/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2334: ordinal not in range(128)

I came across this while trying to make use of the Vagrantfile in brozzler.

This minor change fixes the UnicodeDecodeError for me.

galgeek commented 3 years ago

I can't replicate this issue, likely because my terminal locale setting is UTF-8. @ldko does setting your terminal's locale fix the issue for you?

ldko commented 3 years ago

Hi Barbara, I checked my locale, which looked like:

$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C

Setting LC_ALL with the following removed the UnicodeDecodeError:

$ export LC_ALL="en_US.UTF-8"

Thanks for the assistance @galgeek !