Serialization of messages into bytes sometimes gives non-hex values

[n8ur]

Describe the bug I do not know if this is a bug, improper procedure on my part, or improper understanding of the results. The issue is that the serialized message generated by UBXMessage sometimes includes characters that are not in hex format. The message does work when sent to the receiver, and generates an ACK, so this is about representation, I think.

For example the following message for CFG-TP5 generated by UBXMessage:

<UBX(CFG-TP5, tpIdx=0, version=1, reserved0=0, antCableDelay=50, rfGroupDelay=0, freqPeriod=1, freqPeriodLock=4687500, pulseLenRatio=0, pulseLenRatioLock=50, userConfigDelay=0, active=1, lockGnssFreq=1, lockedOtherSet=1, isFreq=1, isLength=0, alignToTow=1, polarity=1, gridUtcGnss=1, syncMode=0)>

when run through serialize() returns this:

b'\xb5b\x061 \x00\x00\x01\x00\x002\x00\x00\x00\x01\x00\x00\x00\x8c\x86G\x00\x00\x00\x00\x002\x00\x00\x00\x00\x00\x00\x00\xef\x00\x00\x00\x05n'

Note the "\x86G" in the middle and the "\x05n" at the end. I believe that in those cases the letters represent the ASCII representation rather the the hex of the byte.

Again, this message works so it is not a pyubx2 bug, but maybe unexpected behavior that the printed output is mainly hex but includes occasional non-hex representations?

I am trying to generate a byte stream that can be used by another program to send the command, and this quirk is throwing me off. Is there a different way to generate a stream of bytes purely in hex form from the output of UBXMessage?

This is on pyubx2 version 1.2.19 running with Python v3.10.6 on a Linux mint system.

Thanks!

[semuadmin]

Hi @n8ur ,

Thanks for getting in touch. So the behaviour you're seeing is just a quirk of Python standard output. Basically, when Python renders a stream of bytes to standard output (i.e. the terminal), any bytes which happen to represent ASCII characters get rendered as that character, rather than the raw hexadecimal value.

So a value of b'\x62' gets rendered as the letter 'b'. A value of b'\x31' gets rendered as the digit '1'. A value of b'\x20' gets rendered as a space, and so on.

If you're sending the byte stream to a program which is expecting bytes (as opposed to a string), you can simply ignore this - the program will interpet the bytes correctly.

But if you want to see the raw hexadecimal without such rendering, you could use Python's standard hex() function.

pyubx2 also offers its own hextable() utility function which renders a bytestream in tabular hexadecimal format, similar to that used by hex editors.

Here are some examples...

>>> from pyubx2 import UBXMessage, SET, hextable
>>> msg = UBXMessage('CFG','CFG-TP5', SET, tpIdx=0, version=1, antCableDelay=50, rfGroupDelay=0, freqPeriod=1, freqPeriodLock=4687500, pulseLenRatio=0, pulseLenRatioLock=50, userConfigDelay=0, active=1, lockGnssFreq=1, lockedOtherSet=1, isFreq=1, isLength=0, alignToTow=1, polarity=1, gridUtcGnss=1, syncMode=0)

>>> print(msg)
<UBX(CFG-TP5, tpIdx=0, version=1, reserved0=0, antCableDelay=50, rfGroupDelay=0, freqPeriod=1, freqPeriodLock=4687500, pulseLenRatio=0, pulseLenRatioLock=50, userConfigDelay=0, active=1, lockGnssFreq=1, lockedOtherSet=1, isFreq=1, isLength=0, alignToTow=1, polarity=1, gridUtcGnss=1, syncMode=0)>

>>> print(msg.serialize())
b'\xb5b\x061 \x00\x00\x01\x00\x002\x00\x00\x00\x01\x00\x00\x00\x8c\x86G\x00\x00\x00\x00\x002\x00\x00\x00\x00\x00\x00\x00\xef\x00\x00\x00\x05n'

>>> print (msg.serialize().hex())
b562063120000001000032000000010000008c864700000000003200000000000000ef000000056e

>>> print(hextable(msg.serialize()))
000: b562 0631 2000 0001 0000 3200 0000 0100  | b'\xb5b\x061 \x00\x00\x01\x00\x002\x00\x00\x00\x01\x00' |
016: 0000 8c86 4700 0000 0000 3200 0000 0000  | b'\x00\x00\x8c\x86G\x00\x00\x00\x00\x002\x00\x00\x00\x00\x00' |
032: 0000 ef00 0000 056e                      | b'\x00\x00\xef\x00\x00\x00\x05n' |

Hope this helps

[n8ur]

Thanks! I suspected that was what was happening. Thanks for confirming, and also for the way to address it. You can close this issue. Thanks for your very speedy response!