5.6.0 has issues with decoding (or perhaps encoding) bytes

itamarst commented 10 months ago

Things to check first

[X] I have searched the existing issues and didn't find my bug already reported there
[X] I have checked that my bug is still present in the latest release

cbor2 version

5.6.0

Python version

3.11

What happened?

Python 3.11.7 (main, Dec  8 2023, 18:56:58) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cbor2
>>> cbor2.loads(cbor2.dumps(b"\x12" * 65538))
_cbor2.CBORDecodeEOF: premature end of stream (expected to read 65536 bytes, got 2 instead)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: <built-in function loads> returned a result with an exception set

How can we reproduce the bug?

See above.

meejah commented 10 months ago

Pretty sure this is an encoding bug; it encodes the length "65538" as 00 01 00 04 in the tests I had.

meejah commented 10 months ago

Further experimentation leads me to believe I'm wrong above: encoding 65540 bytes with PyPy3 (7.3.12) leads to a decoding error with Python 3.11.5 (whereas decoding with PyPy3 works).

agronholm commented 10 months ago

The new decoding code was a bit more complex than I was comfortable with, so there's probably a bug in there somewhere.

agronholm / cbor2