pynetworktables initialize throws errors about utf-8 encoding

[PeterMitrano]

I tried this:

logging.basicConfig(level=logging.DEBUG)
NetworkTables.initialize(server="10.6.66.5")

And got this:

INFO:nt:NetworkTables initialized in client mode
DEBUG:nt.th:Started thread nt-dispatch-thread-0
DEBUG:nt.th:Started thread nt-client-thread-0
DEBUG:nt:client connected
DEBUG:nt.th:Started thread nt-net-write-0
DEBUG:nt.th:Started thread nt-net-read-0
ERROR:nt:Unhandled exception during handshake
Traceback (most recent call last):
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\network_connection.py", line 248, in _readThreadMain
    handshake_success = self.m_handshake(self, _getMessage, self._sendMessages)
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\dispatcher.py", line 571, in _clientHandshake
    msg = get_msg()
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\network_connection.py", line 236, in _getMessage
    return Message.read(self.m_stream, decoder, self.m_get_entry_type)
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\message.py", line 123, in read
    value = codec.read_value(value_type, rstream)
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\wire.py", line 114, in read_value
    return Value.makeString(self.read_string(rstream))
  File "C:\Users\peter\Anaconda3\lib\site-packages\ntcore\wire.py", line 199, in read_string_v3
    return rstream.read(slen).decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xea in position 9: invalid continuation byte
INFO:nt:DISCONNECTED 10.6.66.5 port 1735 (Robot)
DEBUG:nt.th:Thread nt-net-read-0 exited
DEBUG:nt.th:Thread nt-net-write-0 exited

[virtuald]

What version of python is it? And is it the latest version of pynetworktables?

[virtuald]

Also, what are you connecting to? LabVIEW, Java, C++ ... and what version?

[virtuald]

This looks very similar to https://github.com/robotpy/pynetworktables/issues/42, are you sure you're not using an old version?

[PeterMitrano]

I'm using pynetworktables==2018.1.0 with python 3.5 the network tables server is on the robot, and the robot program is in C++

[PeterMitrano]

I can't reproduce it locally just with outline viewer in server mode, so it's possible the network table running on the robot is somehow out of date. I know it has one of the valid releases of wpilib from this season

[virtuald]

It's most likely a weird key that's being incorrectly encoded that's causing issues. Are you using camera server? Any chance you can connect with outline viewer and look at the camera name or description and see if there's something weird in it?

[virtuald]

I wrote a stream debugger last year... never pushed it to master for some reason. I've updated it and pushed it to the handshake-2018 branch. Please install that... if an exception is thrown, then it will write the stream to a file called 'file.bin'. Attach that here and I'll take a look tomorrow.

[virtuald]

Let me know when I can expect to receive a file.bin from you, and I'll allocate some time in my schedule to take a look at it.

[PeterMitrano]

I'm gonna try this out now--thanks for the quick response

[virtuald]

Any news?

A similar bug has been reported on CD: https://www.chiefdelphi.com/forums/showthread.php?p=1754568

[PeterMitrano]

No I realized I need the physical robot to test and haven't had a chance

On Thu, Apr 5, 2018, 23:52 Dustin Spicuzza notifications@github.com wrote:

Any news?

A similar bug has been reported on CD: https://www.chiefdelphi.com/forums/showthread.php?p=1754568

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/robotpy/pynetworktables/issues/61#issuecomment-379141064, or mute the thread https://github.com/notifications/unsubscribe-auth/AD0zEgdXnXIdLnr32lyEHuIvWw3ffiKfks5tluZ-gaJpZM4S8PFy .

[Kython89]

In 2481's camera name there is a circle R (~restricted~ registered) for the Microsoft live cam. Would that cause problems?

[auscompgeek]

That should be handled correctly, but it's possible it isn't.

(BTW, ® is the registered trademark symbol, not restricted.)

[auscompgeek]

I cannot reproduce a crash by putting a string containing ® in OutlineViewer in server mode and connecting pynetworktables (both Python 2 and Python 3) to it, at least.

[Kython89]

Okay. That was the only thing that stood out as strange to me in the keys / values.

I already sent this Dustin but I'll include it here too. file.bin.zip

[virtuald]

Sorry for the delay, I was at a competition tonight and then stuff afterwards. I'm taking a look now, I'm able to reproduce the crash by sending the file through.

[auscompgeek]

Update: from the file.bin that @Kython89 attached, we have concluded that the robot code is corrupting the value at key /LiveWindow/DriveTrain/.default from 'DriveWithJoystickCommandV2' to b'\x00\xa7>\x03eWithJoystickCommandV2'. Somehow. Not sure how that's happening, could be a WPILibC bug. We'll keep digging.

[virtuald]

For those interested, somehow the memory of a string was being corrupted and passed into NetworkTables (this is still under investigation). As a result, the string received by pynetworktables was not valid utf-8 (as the NT spec dictates), which caused the issue.