Kernel gets blocked and doesn't reply correctly when Python part gets a UnicodeDecodeError

[Naereen]

Hello there, long time fan and user of this awesome kernel speaking.

I've been using this kernel a lot, and something annoys me greatly. Apparently, when I work for a few minutes in ocaml-jupyter, if I get a decoding error (giving this traceback),

[E 23:05:37.177 NotebookApp] Exception in callback functools.partial(<function ZMQStream._update_handler.<locals>.<lambda> at 0x7fdfae544ea0>)
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/tornado/ioloop.py", line 741, in _run_callback
        ret = callback()
      File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 548, in <lambda>
        self.io_loop.add_callback(lambda : self._handle_events(self.socket, 0))
      File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 462, in _handle_events
        self._handle_recv()
      File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 492, in _handle_recv
        self._run_callback(callback, msg)
      File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 444, in _run_callback
        callback(*args, **kwargs)
      File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 203, in <lambda>
        self.on_recv(lambda msg: callback(self, msg), copy=copy)
      File "/usr/local/lib/python3.6/dist-packages/notebook/services/kernels/handlers.py", line 317, in _on_zmq_reply
        msg = self.session.deserialize(fed_msg_list)
      File "/usr/local/lib/python3.6/dist-packages/jupyter_client/session.py", line 937, in deserialize
        message['content'] = self.unpack(msg_list[4])
      File "/usr/local/lib/python3.6/dist-packages/jupyter_client/session.py", line 84, in <lambda>
        json_unpacker = lambda s: jsonapi.loads(s)
      File "/usr/local/lib/python3.6/dist-packages/zmq/utils/jsonapi.py", line 54, in loads
        s = s.decode('utf8')
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 267: invalid continuation byte

then my notebook appears to be non-broken, it display no warning at all, and ocaml cells can be executed, but no output is given! See this GIF:

It's not so bad for me, because now I understood the issue, and simply have to restart the kernel, or sometimes have to kill and restart the underlying jupyter-notebook process... but it takes time, and after that I have to re-execute all the previous cells...

I've not yet been able to understand what characters in the input cell text generates this UnicodeDecodeError, as the traceback shows me that UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc2 in position 267: invalid continuation byte and chr(0xc2) = chr(160) = 'Â' and despite being French I'm sure I didn't write  (never used this letter ever in my life!).

I suspect that the errors, in my case, come from wrongly inserted non-ASCII "non-breakable space", that my keyboard inserts when I (wrongly) type "alt gr + spaceKey"...

But no matter the origin of this decoding issue (from Python side), I'm almost sure that the fact that it breaks (weirdly) the ocaml-jupyter kernel is a bug on the OCaml side. Indeed, I use even more frequently Python notebooks (and even using IJava) and never encountered this blocking bug on either kernels.

I'm looking forward to understanding this, and solving it if possible. Thanks in advance! @Naereen

[akabe]

Thanks for reporting the issue. I haven't seen this error ever. Could you tell me how to reproduce your problem?

[Naereen]

Hi @akabe, Thanks for your quick reply (as always!).

That's my problem, I couldn't find exactly which characters generate this decoding issue... I'll try to keep a close eye on this issue, and copy-paste any blocking cell if I encounter another one...

[akabe]

OK. I wait that you find a cause of this issue. The following command for printing debug logs might help you.

OCAML_JUPYTER_LOG=debug jupyter notebook

[Naereen]

Hi again @akabe, I tried again yesterday on the same notebook, but couldn't trigger the issue again... sadly! I'll try again!

I suspect the error was coming from my keyboard inserting a non-ASCII-128 unbreakable space when using "alt gr + space", but I've been able to fix this on my laptop.