avwx-rest / avwx-engine

Aviation Weather parsing engine. METAR & TAF
MIT License
90 stars 31 forks source link

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 6091107: character maps to <undefined> #31

Closed morphatic closed 4 years ago

morphatic commented 4 years ago

Hi there,

I was trying to follow the basic example on the README of how to use this package. But I got the error:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 6091107: character maps to <undefined>

(Full stack trace below)

Steps to Reproduce

  1. Created a new directory using Git Bash on Windows 10
  2. python -m venv venv
  3. source ./venv/Scripts/activate
  4. pip install avwx-engine
  5. started a python repl with python
  6. typed:
    >>> import avwx
    >>> metar = avwx.Metar('KJFK')
  7. Immediately got an error

Full Stack Trace

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\morga\dev\avwx-test\venv\lib\site-packages\avwx\current\base.py", line 65, in __init__
    super().__init__(icao)
  File "C:\Users\morga\dev\avwx-test\venv\lib\site-packages\avwx\base.py", line 61, in __init__
    self.station = Station.from_icao(icao)
  File "C:\Users\morga\dev\avwx-test\venv\lib\site-packages\avwx\station.py", line 158, in from_icao
    info = copy(_STATIONS[ident.upper()])
  File "C:\Users\morga\dev\avwx-test\venv\lib\site-packages\avwx\structs.py", line 29, in __getitem__
    self._load()
  File "C:\Users\morga\dev\avwx-test\venv\lib\site-packages\avwx\structs.py", line 25, in _load
    self.data = json.load(self.source.open())
  File "C:\Python38\lib\json\__init__.py", line 293, in load
    return loads(fp.read(),
  File "C:\Python38\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 6091107: character maps to <undefined>

Am I doing something wrong?

devdupont commented 4 years ago

This appears to be limited to Windows as I haven't done excessive testing on that platform. Can test on mine this weekend and push an update.

My initial guess is that Windows defaults the JSON encoding to something other than UTF-8 which the station details file uses extensively. For the time being, changing: File "C:\Users\morga\dev\avwx-test\venv\lib\site-packages\avwx\structs.py", line 25 to: self.data = json.load(self.source.open(encoding="utf8")) should fix this issue in the meantime.

morphatic commented 4 years ago

That worked! Thanks for the quick reply!!! For my part, feel free to close this issue. Since you said you wanted to do more testing, I'll let you do it in case you want this here as a reminder.