Problems with msgpack 1.0.0

zacharyneveu commented 4 years ago

While working on a different package I updated msgpack to version 1.0.0, and accidentally broke all of my Neovim plugins. Reverting back to v0.6.2 fixes the problem, so I'm assuming there was an API change. Not sure if this requires immediate work or not, but thought I'd mention it. Below is the output of :messages in Neovim to show what's going on.

[ncm2_core@yarp] Traceback (most recent call last):
[ncm2_core@yarp]   File "/home/zach/.config/nvim/plugged/nvim-yarp/pythonx/yarp.py", line 52, in <module>
[ncm2_core@yarp]     nvim = attach('socket', path=serveraddr)
[ncm2_core@yarp]   File "/home/zach/.local/lib/python3.7/site-packages/pynvim/__init__.py", line 111, in attach
[ncm2_core@yarp]     socket_session(path) if session_type == 'socket' else
[ncm2_core@yarp]   File "/home/zach/.local/lib/python3.7/site-packages/pynvim/msgpack_rpc/__init__.py", line 32, in soc
ket_session
[ncm2_core@yarp]     return session('socket', path)
[ncm2_core@yarp]   File "/home/zach/.local/lib/python3.7/site-packages/pynvim/msgpack_rpc/__init__.py", line 19, in ses
sion
[ncm2_core@yarp]     msgpack_stream = MsgpackStream(loop)
[ncm2_core@yarp]   File "/home/zach/.local/lib/python3.7/site-packages/pynvim/msgpack_rpc/msgpack_stream.py", line 24, 
in __init__
[ncm2_core@yarp]     unicode_errors=unicode_errors_default)
[ncm2_core@yarp]   File "msgpack/_packer.pyx", line 118, in msgpack._cmsgpack.Packer.__init__
[ncm2_core@yarp] TypeError: __init__() got an unexpected keyword argument 'encoding'
[ncm2_core@yarp] During handling of the above exception, another exception occurred:
[ncm2_core@yarp] Traceback (most recent call last):
[ncm2_core@yarp]   File "/home/zach/.config/nvim/plugged/nvim-yarp/pythonx/yarp.py", line 71, in <module>
[ncm2_core@yarp]     nvim.close()
[ncm2_core@yarp] AttributeError: 'NoneType' object has no attribute 'close'
[ncm2_core@yarp] Job is dead. cmd=['/home/zach/anaconda3_v2/bin/python', '-u', '/home/zach/.config/nvim/plugged/nvim-ya
rp/pythonx/yarp.py', '/tmp/nvimMMvEn6/0', 2, 'ncm2_core']

Shougo commented 4 years ago

https://github.com/neovim/pynvim/blob/95e6995ea92e97ed906cb22e422f3e591c72f9c5/pynvim/msgpack_rpc/msgpack_stream.py#L23

Please read the code and the commit.

https://github.com/neovim/pynvim/commit/c771bfd4c13683096d10432376a9dedb4aef66b2

Your problem is already fixed in the latest pynvim. So your pynvim is too old.

somini commented 4 years ago

I still have problems with msgpack v1. Should I open a new issue?

This hanged nvim completely on hitting backspace in a denite buffer (I'm on the old Shougo/denite.nvim@67475c7, before the v3 rewrite).

Here's a traceback:

2020-04-06 22:09:23,683 [DEBUG @ session.py:_yielding_request:163] 6621 - yielding from greenlet <greenlet.greenlet object at 0x7fbc9aaf0400> to wait for response
2020-04-06 22:09:23,683 [DEBUG @ msgpack_stream.py:_on_data:58] 6621 - waiting for message...
2020-04-06 22:09:23,683 [DEBUG @ msgpack_stream.py:_on_data:63] 6621 - unpacker needs more data...
2020-04-06 22:09:23,684 [DEBUG @ msgpack_stream.py:_on_data:58] 6621 - waiting for message...
2020-04-06 22:09:23,684 [DEBUG @ msgpack_stream.py:_on_data:60] 6621 - received message: [1, 1555, None, None]
2020-04-06 22:09:23,685 [DEBUG @ async_session.py:_on_response:100] 6621 - received response: None, None
2020-04-06 22:09:23,685 [DEBUG @ session.py:response_cb:159] 6621 - response is available for greenlet <greenlet.greenlet object at 0x7fbc9aaf0400>, switching back
2020-04-06 22:09:23,685 [DEBUG @ msgpack_stream.py:send:33] 6621 - sent [0, 1556, 'nvim_call_function', ('denite#util#getchar', [False])]
2020-04-06 22:09:23,685 [DEBUG @ base.py:send:118] 6621 - Sending 'b'\x94\x00\xcd\x06\x14\xb2nvim_call_function\x92\xb3denite#util#getchar\x91\xc2''
2020-04-06 22:09:23,685 [DEBUG @ session.py:_yielding_request:163] 6621 - yielding from greenlet <greenlet.greenlet object at 0x7fbc9aaf0400> to wait for response
2020-04-06 22:09:23,685 [DEBUG @ msgpack_stream.py:_on_data:58] 6621 - waiting for message...
2020-04-06 22:09:23,686 [DEBUG @ msgpack_stream.py:_on_data:63] 6621 - unpacker needs more data...
2020-04-06 22:09:23,686 [DEBUG @ msgpack_stream.py:_on_data:58] 6621 - waiting for message...
2020-04-06 22:09:23,686 [ERROR @ base_events.py:default_exception_handler:1707] 6621 - Exception in callback _UnixReadPipeTransport._read_ready()
handle: <Handle _UnixReadPipeTransport._read_ready()>
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 500, in _read_ready
    self._protocol.data_received(data)
  File "/usr/lib/python3.8/site-packages/pynvim/msgpack_rpc/event_loop/asyncio.py", line 60, in data_received
    self._on_data(data)
  File "/usr/lib/python3.8/site-packages/pynvim/msgpack_rpc/msgpack_stream.py", line 59, in _on_data
    msg = next(self._unpacker)
  File "msgpack/_unpacker.pyx", line 528, in msgpack._cmsgpack.Unpacker.__next__
  File "msgpack/_unpacker.pyx", line 459, in msgpack._cmsgpack.Unpacker._unpack
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

I have the latest v0.4.1 on the latest Arch Linux:

python 3.8.2-1
python-pynvim 0.4.1-1
python-msgpack 1.0.0
neovim 0.4.3-3

Downgrading to `python-msgpack` v0.6.2 resolves it.

From https://pypi.org/project/msgpack/#description and https://github.com/msgpack/msgpack-python/commit/5534d0c7af0114db3d27f7b96c82a7fe22ce1e40#diff-88b99bb28683bd5b7e3a204826ead112R40, I'm guessing the Unpacker needs the raw_as_bytes option to keep the same behaviour as before?

Shougo commented 4 years ago

I still have problems with msgpack v1. Should I open a new issue?

You should open new issue. Because, it is not same problem.

neovim / pynvim