0.2.3 regression: struct.error: byte format requires -128 <= number <= 127

[voetsjoeba]

Here's an xxd dump of a file that passes a loads/dumps roundtrip in 0.2.2, but fails in 0.2.3:

0000000: aced 0005 7372 0033 636f 6d2e 7375 6e2e  ....sr.3com.sun.
0000010: 6372 7970 746f 2e70 726f 7669 6465 722e  crypto.provider.
0000020: 5365 616c 6564 4f62 6a65 6374 466f 724b  SealedObjectForK
0000030: 6579 5072 6f74 6563 746f 72cd 57ca 59e7  eyProtector.W.Y.
0000040: 30bb 5302 0000 7872 0019 6a61 7661 782e  0.S...xr..javax.
0000050: 6372 7970 746f 2e53 6561 6c65 644f 626a  crypto.SealedObj
0000060: 6563 743e 363d a6c3 b754 7002 0004 5b00  ect>6=...Tp...[.
0000070: 0d65 6e63 6f64 6564 5061 7261 6d73 7400  .encodedParamst.
0000080: 025b 425b 0010 656e 6372 7970 7465 6443  .[B[..encryptedC
0000090: 6f6e 7465 6e74 7100 7e00 024c 0009 7061  ontentq.~..L..pa
00000a0: 7261 6d73 416c 6774 0012 4c6a 6176 612f  ramsAlgt..Ljava/
00000b0: 6c61 6e67 2f53 7472 696e 673b 4c00 0773  lang/String;L..s
00000c0: 6561 6c41 6c67 7100 7e00 0378 7075 7200  ealAlgq.~..xpur.
00000d0: 025b 42ac f317 f806 0854 e002 0000 7870  .[B......T....xp
00000e0: 0000 000f 300d 0408 4b28 2520 e440 5a01  ....0...K(% .@Z.
00000f0: 0201 1475 7100 7e00 0500 0001 08e4 50fa  ...uq.~.......P.
0000100: 6ccd 4c40 e0db 498b 57ab fcf0 09bc 3ac8  l.L@..I.W.....:.
0000110: 0608 1915 aad3 77de ed45 e8b6 5c77 fe60  ......w..E..\w.`
0000120: 6eb5 3cd2 05fd 7701 5d93 f5b1 6412 ff0a  n.<...w.]...d...
0000130: 6b67 2a18 106f fe82 097a c5a8 dd95 f2dc  kg*..o...z......
0000140: 4250 a635 b3ed 78ab 54ed ad88 dd95 4508  BP.5..x.T.....E.
0000150: 997e 7dc9 b140 9a5f f0d4 a3f3 8bc1 b58b  .~}..@._........
0000160: d93b 9d24 1017 81fa 5582 193d 8133 c9b9  .;.$....U..=.3..
0000170: 5f07 99e5 7829 762e b2e2 49db 4a45 d561  _...x)v...I.JE.a
0000180: 064f 6daa 1fca 5159 462f 8c2d 873c 5c95  .Om...QYF/.-.<\.
0000190: e98c 56f9 f4fe df75 593a ef9e 2c11 b03a  ..V....uY:..,..:
00001a0: 0a7e a94b e2fa 5c06 884c b91b 53b3 57a0  .~.K..\..L..S.W.
00001b0: 5938 4e30 a015 8869 5b0e f7c5 a308 4603  Y8N0...i[.....F.
00001c0: 1579 affb 0b29 0a1a de8b 9c56 83e2 de65  .y...).....V...e
00001d0: ab22 537e be4a fe04 2a14 55cf abb6 3838  ."S~.J..*.U...88
00001e0: a2bf c384 39b1 c659 3000 b521 dcc3 b8b5  ....9..Y0..!....
00001f0: 676f 07e9 879c 4883 c86d 9235 655e f131  go....H..m.5e^.1
0000200: a915 5936 d574 0016 5042 4557 6974 684d  ..Y6.t..PBEWithM
0000210: 4435 416e 6454 7269 706c 6544 4553 7400  D5AndTripleDESt.
0000220: 1650 4245 5769 7468 4d44 3541 6e64 5472  .PBEWithMD5AndTr
0000230: 6970 6c65 4445 53                        ipleDES

This is on Python 2.7.5 on a CentOS 6 box. Roundtrip script that I used:

#!/bin/env python
import javaobj

with open("input.ser", "rb") as input_file:
        with open("output_%s.ser" % javaobj.__version__, "wb") as output_file:
                obj = javaobj.load(input_file)
                output_file.write(javaobj.dumps(obj))

Produces the following error on 0.2.3:

Traceback (most recent call last):
  File "./redump.py", line 7, in <module>
    output_file.write(javaobj.dumps(obj))
  File "/home/vmuser/py27env/lib/python2.7/site-packages/javaobj.py", line 209, in dumps
    return marshaller.dump(obj)
  File "/home/vmuser/py27env/lib/python2.7/site-packages/javaobj.py", line 1167, in dump
    self.writeObject(obj)
  File "/home/vmuser/py27env/lib/python2.7/site-packages/javaobj.py", line 1192, in writeObject
    self.write_object(obj)
  File "/home/vmuser/py27env/lib/python2.7/site-packages/javaobj.py", line 1372, in write_object
    self._write_value(field_type, getattr(obj, field_name))
  File "/home/vmuser/py27env/lib/python2.7/site-packages/javaobj.py", line 1525, in _write_value
    self.write_array(value)
  File "/home/vmuser/py27env/lib/python2.7/site-packages/javaobj.py", line 1492, in write_array
    self._write_value(type_char, v)
  File "/home/vmuser/py27env/lib/python2.7/site-packages/javaobj.py", line 1508, in _write_value
    self._writeStruct(">b", 1, (value,))
  File "/home/vmuser/py27env/lib/python2.7/site-packages/javaobj.py", line 1218, in _writeStruct
    ba = struct.pack(unpack, *args)
struct.error: byte format requires -128 <= number <= 127

[tcalmant]

OK, bug reproduced, I'll take a look into it

[tcalmant]

The problem comes from the use of bytearray in JavaByteArray, as it only accepts integers between 0 and 256, whereas Java bytes are signed, between -128 and 127.

[tcalmant]

Could you test with the latest commit on master ? (commit 91edde8)

[voetsjoeba]

Yep, appears to work. Produces an identical output file to 0.2.2 now. EDIT: Although now an existing test case appears to be failing according to the Travis CI output

[tcalmant]

Yeah, the Travis test is based on the bytearray behaviour, I'll correct it later, before releasing 0.2.4.

[tcalmant]

This issue seems to be fixed, please reopen it if necessary