search

jmelesky / omwllf

OpenMW leveled list fixer
ISC License
38 stars 9 forks source link

UnicodeDecodeError #1

Closed jmelesky closed 7 years ago

jmelesky commented 7 years ago

Reported via irc (#openmw on freenode).

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcf in position 140: invalid continuation byte" File "./omwllf.py", line 485, in <module>

File "./omwllf.py", line 41, in parseString

Platform info:

Up-To_date Arch.
LC_ADDRESS=C
LC_COLLATE=C
LC_CTYPE=en_US.UTF-8
LC_IDENTIFICATION=C
LC_MEASUREMENT=C
LC_MESSAGES=C
LC_MONETARY=C
LC_NAME=C
LC_NUMERIC=C
LC_PAPER=C
LC_TELEPHONE=C
LC_TIME=C

Probably during processing of Robert's Bodies.

jmelesky commented 7 years ago

The mod in question has a bunch of non-ascii, non-unicode stuff in the mod description field. The python default behavior for bytes.decode() involves throwing exceptions on non-compliant strings.

I've updated to parse only ascii, and to ignore non-compliant characters, which should fix this.