search

eastein / andrey

markov chain fun
7 stars 0 forks source link

python 3 compatibility #11

Open eastein opened 8 years ago

eastein commented 8 years ago

Got started in a py3 labeled branch which I have since tagged (0.1.0) and released on PyPI. More work to do - under python 3.5.2, I get this when trying to start the bot (which really is just reading a msgpack persisted chain that was produced under python 2.7.12, using the same revision of the code) - the issue lies within andrey, not andrey_bot.

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/andrey_bot/run.py", line 186, in <module>
    ratio=float(options.ratio), word_replace=word_replace, word_filter=word_filter, read_only=options.read_only, chatty=options.chatty, chain_length=int(options.chain_length))
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/andrey_bot/run.py", line 139, in __init__
    self.m = persist.PersistedMarkov.restore(filename, 2, 3)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/andrey/persist.py", line 53, in restore
    return cls.fromdict(next(msgpack.Unpacker(open(filename), encoding='utf-8')))
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/msgpack/fallback.py", line 557, in next
    ret = self._fb_unpack(EX_CONSTRUCT, None)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/msgpack/fallback.py", line 498, in _fb_unpack
    typ, n, obj = self._read_header(execute, write_bytes)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/msgpack/fallback.py", line 346, in _read_header
    c = self._fb_read(1, write_bytes)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/msgpack/fallback.py", line 324, in _fb_read
    tmp = self.file_like.read(to_read)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 0: invalid start byte
eastein commented 8 years ago

I started doing some research into this and found that there have been compatiblity changes in msgpack-python at different points around encoding, which may be playing poorly with python 3's own encoding compatibility issues: https://pypi.python.org/pypi/msgpack-python - see section "Notes / string and binary type".

eastein commented 8 years ago

I will want to use tox to automate testing while working on this issue. https://tox.readthedocs.io/en/latest/

estein-de commented 1 year ago

On python 3.9.16, on macOS, using pyenv-virtualenv and having installed 0.1.0 off of PyPI, I get this when trying to train a chain:

$ echo "test" | python  -m andrey.train test.mpack 2 3
test.mpack 2 3
Traceback (most recent call last):
  File "/Users/EStein/.pyenv/versions/3.9.16/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/EStein/.pyenv/versions/3.9.16/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/EStein/.pyenv/versions/andrey-3916/lib/python3.9/site-packages/andrey/train.py", line 25, in <module>
    m.save(fn)
  File "/Users/EStein/.pyenv/versions/andrey-3916/lib/python3.9/site-packages/andrey/persist.py", line 31, in save
    msgpack.dump(self.todict(), fh)
  File "/Users/EStein/.pyenv/versions/andrey-3916/lib/python3.9/site-packages/msgpack/__init__.py", line 38, in pack
    stream.write(packer.pack(o))
TypeError: write() argument must be str, not bytes