search

kbandla / dpkt

fast, simple packet creation / parsing, with definitions for the basic TCP/IP protocols
Other
1.09k stars 270 forks source link

Seven tests fail on s390x #505

Closed mcepl closed 2 years ago

mcepl commented 3 years ago

When building package for dpkt for OpenSUSE/Factory, I get these tests to fail on s390x, which has very weird hardware. Complete build log

[   27s] =================================== FAILURES ===================================
[   27s] __________________________________ test_utils __________________________________
[   27s] 
[   27s]     def test_utils():
[   27s]         __buf = b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e'
[   27s]         __hd = '  0000:  00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e     ...............'
[   27s]         h = hexdump(__buf)
[   27s]         assert (h == __hd)
[   27s]         c = in_cksum(__buf)
[   27s] >       assert (c == 51150)
[   27s] E       assert 54720 == 51150
[   27s] E         +54720
[   27s] E         -51150
[   27s] 
[   27s] dpkt/dpkt.py:218: AssertionError
[   27s] __________________________________ test_utils __________________________________
[   27s] 
[   27s]     def test_utils():
[   27s]         __buf = b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\r\x0e'
[   27s]         __hd = '  0000:  00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e     ...............'
[   27s]         h = hexdump(__buf)
[   27s]         assert (h == __hd)
[   27s]         c = in_cksum(__buf)
[   27s] >       assert (c == 51150)
[   27s] E       assert 54720 == 51150
[   27s] E         +54720
[   27s] E         -51150
[   27s] 
[   27s] dpkt/dpkt.py:218: AssertionError
[   27s] ______________________________ test_80211_beacon _______________________________
[   27s] 
[   27s]     def test_80211_beacon():
[   27s]         s = b'\x80\x00\x00\x00\xff\xff\xff\xff\xff\xff\x00\x26\xcb\x18\x6a\x30\x00\x26\xcb\x18\x6a\x30\xa0\xd0\x77\x09\x32\x03\x8f\x00\x00\x00\x66\x00\x31\x04\x00\x04\x43\x41\x45\x4e\x01\x08\x82\x84\x8b\x0c\x12\x96\x18\x24\x03\x01\x01\x05\x04\x00\x01\x00\x00\x07\x06\x55\x53\x20\x01\x0b\x1a\x0b\x05\x00\x00\x6e\x00\x00\x2a\x01\x02\x2d\x1a\x6e\x18\x1b\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x30\x14\x01\x00\x00\x0f\xac\x04\x01\x00\x00\x0f\xac\x04\x01\x00\x00\x0f\xac\x01\x28\x00\x32\x04\x30\x48\x60\x6c\x36\x03\x51\x63\x03\x3d\x16\x01\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x85\x1e\x05\x00\x8f\x00\x0f\x00\xff\x03\x59\x00\x63\x73\x65\x2d\x33\x39\x31\x32\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x36\x96\x06\x00\x40\x96\x00\x14\x00\xdd\x18\x00\x50\xf2\x02\x01\x01\x80\x00\x03\xa4\x00\x00\x27\xa4\x00\x00\x42\x43\x5e\x00\x62\x32\x2f\x00\xdd\x06\x00\x40\x96\x01\x01\x04\xdd\x05\x00\x40\x96\x03\x05\xdd\x05\x00\x40\x96\x0b\x09\xdd\x08\x00\x40\x96\x13\x01\x00\x34\x01\xdd\x05\x00\x40\x96\x14\x05'
[   27s]         ieee = IEEE80211(s, fcs=True)
[   27s]         assert ieee.version == 0
[   27s]         assert ieee.type == MGMT_TYPE
[   27s]         assert ieee.subtype == M_BEACON
[   27s]         assert ieee.to_ds == 0
[   27s]         assert ieee.from_ds == 0
[   27s]         assert ieee.pwr_mgt == 0
[   27s]         assert ieee.more_data == 0
[   27s]         assert ieee.wep == 0
[   27s]         assert ieee.order == 0
[   27s]         assert ieee.mgmt.dst == b'\xff\xff\xff\xff\xff\xff'
[   27s]         assert ieee.mgmt.src == b'\x00\x26\xcb\x18\x6a\x30'
[   27s]         assert ieee.beacon.capability == 0x3104
[   27s] >       assert ieee.capability.privacy == 1
[   27s] E       assert 0 == 1
[   27s] E         +0
[   27s] E         -1
[   27s] 
[   27s] dpkt/ieee80211.py:676: AssertionError
[   27s] __________________________ test_compressed_block_ack ___________________________
[   27s] 
[   27s] self = BlockAck(dst=b'4\xc0Y\xd6?b', src=b'\xb4u\x0eF\x83\xc1', ctl=1360, seq=33006, data=b'\x03\x00\x00\x00\x00\x00\x00\x00')
[   27s] args = (b'4\xc0Y\xd6?b\xb4u\x0eF\x83\xc1\x05P\x80\xee\x03\x00\x00\x00\x00\x00\x00\x00',)
[   27s] kwargs = {}
[   27s] 
[   27s]     def __init__(self, *args, **kwargs):
[   27s]         """Packet constructor with ([buf], [field=val,...]) prototype.
[   27s]     
[   27s]         Arguments:
[   27s]     
[   27s]         buf -- optional packet buffer to unpack
[   27s]     
[   27s]         Optional keyword arguments correspond to members to set
[   27s]         (matching fields in self.__hdr__, or 'data').
[   27s]         """
[   27s]         self.data = b''
[   27s]         if args:
[   27s]             try:
[   27s] >               self.unpack(args[0])
[   27s] 
[   27s] dpkt/dpkt.py:89: 
[   27s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   27s] 
[   27s] self = BlockAck(dst=b'4\xc0Y\xd6?b', src=b'\xb4u\x0eF\x83\xc1', ctl=1360, seq=33006, data=b'\x03\x00\x00\x00\x00\x00\x00\x00')
[   27s] buf = b'4\xc0Y\xd6?b\xb4u\x0eF\x83\xc1\x05P\x80\xee\x03\x00\x00\x00\x00\x00\x00\x00'
[   27s] 
[   27s]     def unpack(self, buf):
[   27s]         dpkt.Packet.unpack(self, buf)
[   27s]         self.data = buf[self.__hdr_len__:]
[   27s]         self.ctl = socket.ntohs(self.ctl)
[   27s]     
[   27s]         if self.compressed:
[   27s]             self.bmp = struct.unpack('8s', self.data[0:_COMPRESSED_BMP_LENGTH])[0]
[   27s]         else:
[   27s] >           self.bmp = struct.unpack('128s', self.data[0:_BMP_LENGTH])[0]
[   27s] E           struct.error: unpack requires a buffer of 128 bytes
[   27s] 
[   27s] dpkt/ieee80211.py:430: error
[   27s] 
[   27s] During handling of the above exception, another exception occurred:
[   27s] 
[   27s]     def test_compressed_block_ack():
[   27s]         s = b'\x94\x00\x00\x00\x34\xc0\x59\xd6\x3f\x62\xb4\x75\x0e\x46\x83\xc1\x05\x50\x80\xee\x03\x00\x00\x00\x00\x00\x00\x00\xa2\xe4\x98\x45'
[   27s] >       ieee = IEEE80211(s, fcs=True)
[   27s] 
[   27s] dpkt/ieee80211.py:751: 
[   27s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   27s] dpkt/ieee80211.py:279: in __init__
[   27s]     super(IEEE80211, self).__init__(*args, **kwargs)
[   27s] dpkt/dpkt.py:89: in __init__
[   27s]     self.unpack(args[0])
[   27s] dpkt/ieee80211.py:358: in unpack
[   27s]     field = parser(self.data)
[   27s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   27s] 
[   27s] self = BlockAck(dst=b'4\xc0Y\xd6?b', src=b'\xb4u\x0eF\x83\xc1', ctl=1360, seq=33006, data=b'\x03\x00\x00\x00\x00\x00\x00\x00')
[   27s] args = (b'4\xc0Y\xd6?b\xb4u\x0eF\x83\xc1\x05P\x80\xee\x03\x00\x00\x00\x00\x00\x00\x00',)
[   27s] kwargs = {}
[   27s] 
[   27s]     def __init__(self, *args, **kwargs):
[   27s]         """Packet constructor with ([buf], [field=val,...]) prototype.
[   27s]     
[   27s]         Arguments:
[   27s]     
[   27s]         buf -- optional packet buffer to unpack
[   27s]     
[   27s]         Optional keyword arguments correspond to members to set
[   27s]         (matching fields in self.__hdr__, or 'data').
[   27s]         """
[   27s]         self.data = b''
[   27s]         if args:
[   27s]             try:
[   27s]                 self.unpack(args[0])
[   27s]             except struct.error:
[   27s]                 if len(args[0]) < self.__hdr_len__:
[   27s]                     raise NeedData
[   27s] >               raise UnpackError('invalid %s: %r' %
[   27s]                                   (self.__class__.__name__, args[0]))
[   27s] E               dpkt.dpkt.UnpackError: invalid BlockAck: b'4\xc0Y\xd6?b\xb4u\x0eF\x83\xc1\x05P\x80\xee\x03\x00\x00\x00\x00\x00\x00\x00'
[   27s] 
[   27s] dpkt/dpkt.py:93: UnpackError
[   27s] ________________________ test_action_block_ack_request _________________________
[   27s] 
[   27s]     def test_action_block_ack_request():
[   27s]         s = b'\xd0\x00\x3a\x01\x00\x23\x14\x36\x52\x30\xb4\x75\x0e\x46\x83\xc1\xb4\x75\x0e\x46\x83\xc1\x70\x14\x03\x00\x0d\x02\x10\x00\x00\x40\x29\x06\x50\x33\x9e'
[   27s]         ieee = IEEE80211(s, fcs=True)
[   27s]         assert ieee.type == MGMT_TYPE
[   27s]         assert ieee.subtype == M_ACTION
[   27s]         assert ieee.action.category == BLOCK_ACK
[   27s]         assert ieee.action.code == BLOCK_ACK_CODE_REQUEST
[   27s]         assert ieee.action.block_ack_request.timeout == 0
[   27s]         parameters = struct.unpack('H', b'\x10\x02')[0]
[   27s] >       assert ieee.action.block_ack_request.parameters == parameters
[   27s] E       assert 528 == 4098
[   27s] E         +528
[   27s] E         -4098
[   27s] 
[   27s] dpkt/ieee80211.py:770: AssertionError
[   27s] ________________________ test_action_block_ack_response ________________________
[   27s] 
[   27s]     def test_action_block_ack_response():
[   27s]         s = b'\xd0\x00\x3c\x00\xb4\x75\x0e\x46\x83\xc1\x00\x23\x14\x36\x52\x30\xb4\x75\x0e\x46\x83\xc1\xd0\x68\x03\x01\x0d\x00\x00\x02\x10\x88\x13\x9f\xc0\x0b\x75'
[   27s]         ieee = IEEE80211(s, fcs=True)
[   27s]         assert ieee.type == MGMT_TYPE
[   27s]         assert ieee.subtype == M_ACTION
[   27s]         assert ieee.action.category == BLOCK_ACK
[   27s]         assert ieee.action.code == BLOCK_ACK_CODE_RESPONSE
[   27s]         timeout = struct.unpack('H', b'\x13\x88')[0]
[   27s] >       assert ieee.action.block_ack_response.timeout == timeout
[   27s] E       assert 34835 == 5000
[   27s] E         +34835
[   27s] E         -5000
[   27s] 
[   27s] dpkt/ieee80211.py:780: AssertionError
[   27s] ____________________________ test_custom_read_write ____________________________
[   27s] 
[   27s]     def test_custom_read_write():
[   27s]         """Test a full pcapng file with 1 ICMP packet"""
[   27s]         buf = (
[   27s]             b'\x0a\x0d\x0d\x0a\x7c\x00\x00\x00\x4d\x3c\x2b\x1a\x01\x00\x00\x00\xff\xff\xff\xff\xff\xff'
[   27s]             b'\xff\xff\x03\x00\x1e\x00\x36\x34\x2d\x62\x69\x74\x20\x57\x69\x6e\x64\x6f\x77\x73\x20\x38'
[   27s]             b'\x2e\x31\x2c\x20\x62\x75\x69\x6c\x64\x20\x39\x36\x30\x30\x00\x00\x04\x00\x34\x00\x44\x75'
[   27s]             b'\x6d\x70\x63\x61\x70\x20\x31\x2e\x31\x32\x2e\x37\x20\x28\x76\x31\x2e\x31\x32\x2e\x37\x2d'
[   27s]             b'\x30\x2d\x67\x37\x66\x63\x38\x39\x37\x38\x20\x66\x72\x6f\x6d\x20\x6d\x61\x73\x74\x65\x72'
[   27s]             b'\x2d\x31\x2e\x31\x32\x29\x00\x00\x00\x00\x7c\x00\x00\x00\x01\x00\x00\x00\x7c\x00\x00\x00'
[   27s]             b'\x01\x00\x00\x00\x00\x00\x04\x00\x02\x00\x32\x00\x5c\x44\x65\x76\x69\x63\x65\x5c\x4e\x50'
[   27s]             b'\x46\x5f\x7b\x33\x42\x42\x46\x32\x31\x41\x37\x2d\x39\x31\x41\x45\x2d\x34\x44\x44\x42\x2d'
[   27s]             b'\x41\x42\x32\x43\x2d\x43\x37\x38\x32\x39\x39\x39\x43\x32\x32\x44\x35\x7d\x00\x00\x09\x00'
[   27s]             b'\x01\x00\x06\x00\x00\x00\x0c\x00\x1e\x00\x36\x34\x2d\x62\x69\x74\x20\x57\x69\x6e\x64\x6f'
[   27s]             b'\x77\x73\x20\x38\x2e\x31\x2c\x20\x62\x75\x69\x6c\x64\x20\x39\x36\x30\x30\x00\x00\x00\x00'
[   27s]             b'\x00\x00\x7c\x00\x00\x00\x06\x00\x00\x00\x84\x00\x00\x00\x00\x00\x00\x00\x63\x20\x05\x00'
[   27s]             b'\xd6\xc4\xab\x0b\x4a\x00\x00\x00\x4a\x00\x00\x00\x08\x00\x27\x96\xcb\x7c\x52\x54\x00\x12'
[   27s]             b'\x35\x02\x08\x00\x45\x00\x00\x3c\xa4\x40\x00\x00\x1f\x01\x27\xa2\xc0\xa8\x03\x28\x0a\x00'
[   27s]             b'\x02\x0f\x00\x00\x56\xf0\x00\x01\x00\x6d\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c'
[   27s]             b'\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x41\x42\x43\x44\x45\x46\x47\x48\x49\x00\x00'
[   27s]             b'\x01\x00\x0f\x00\x64\x70\x6b\x74\x20\x69\x73\x20\x61\x77\x65\x73\x6f\x6d\x65\x00\x00\x00'
[   27s]             b'\x00\x00\x84\x00\x00\x00')
[   27s]     
[   27s]         fobj = BytesIO(buf)
[   27s]     
[   27s]         # test reading
[   27s]         reader = Reader(fobj)
[   27s]         assert reader.snaplen == 0x40000
[   27s]         assert reader.datalink() == DLT_EN10MB
[   27s]     
[   27s]         assert reader.idb.opts[0].data.decode('utf-8') == '\\Device\\NPF_{3BBF21A7-91AE-4DDB-AB2C-C782999C22D5}'
[   27s]         assert reader.idb.opts[2].data.decode('utf-8') == '64-bit Windows 8.1, build 9600'
[   27s]     
[   27s]         ts, buf1 = next(iter(reader))
[   27s]         assert ts == 1442984653.2108380
[   27s]         assert len(buf1) == 74
[   27s]     
[   27s]         assert buf1.startswith(b'\x08\x00\x27\x96')
[   27s]         assert buf1.endswith(b'FGHI')
[   27s]         fobj.close()
[   27s]     
[   27s]         # test pcapng customized writing
[   27s]         shb = SectionHeaderBlockLE(opts=[
[   27s]             PcapngOptionLE(code=3, data=b'64-bit Windows 8.1, build 9600'),
[   27s]             PcapngOptionLE(code=4, data=b'Dumpcap 1.12.7 (v1.12.7-0-g7fc8978 from master-1.12)'),
[   27s]             PcapngOptionLE()
[   27s]         ])
[   27s]         idb = InterfaceDescriptionBlockLE(snaplen=0x40000, opts=[
[   27s]             PcapngOptionLE(code=2, data=b'\\Device\\NPF_{3BBF21A7-91AE-4DDB-AB2C-C782999C22D5}'),
[   27s]             PcapngOptionLE(code=9, data=b'\x06'),
[   27s]             PcapngOptionLE(code=12, data=b'64-bit Windows 8.1, build 9600'),
[   27s]             PcapngOptionLE()
[   27s]         ])
[   27s]         epb = EnhancedPacketBlockLE(opts=[
[   27s]             PcapngOptionLE(code=1, text=b'dpkt is awesome'),
[   27s]             PcapngOptionLE()
[   27s]         ], pkt_data=(
[   27s]             b'\x08\x00\x27\x96\xcb\x7c\x52\x54\x00\x12\x35\x02\x08\x00\x45\x00\x00\x3c\xa4\x40\x00\x00'
[   27s]             b'\x1f\x01\x27\xa2\xc0\xa8\x03\x28\x0a\x00\x02\x0f\x00\x00\x56\xf0\x00\x01\x00\x6d\x41\x42'
[   27s]             b'\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x41'
[   27s]             b'\x42\x43\x44\x45\x46\x47\x48\x49'
[   27s]         ))
[   27s]         fobj = BytesIO()
[   27s] >       writer = Writer(fobj, shb=shb, idb=idb)
[   27s] 
[   27s] dpkt/pcapng.py:729: 
[   27s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   27s] dpkt/pcapng.py:318: in __init__
[   27s]     self._validate_block('shb', shb, SectionHeaderBlock)
[   27s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   27s] 
[   27s] self = <dpkt.pcapng.Writer object at 0x3ff86b74460>, arg_name = 'shb'
[   27s] blk = SectionHeaderBlockLE(opts=[PcapngOptionLE(code=3, data=b'64-bit Windows 8.1, build 9600'), PcapngOptionLE(code=4, data=b'Dumpcap 1.12.7 (v1.12.7-0-g7fc8978 from master-1.12)'), PcapngOptionLE(opt_endofopt)])
[   27s] expected_cls = <class 'dpkt.pcapng.SectionHeaderBlock'>
[   27s] 
[   27s]     def _validate_block(self, arg_name, blk, expected_cls):
[   27s]         """Check a user-defined block for correct type and endianness"""
[   27s]         if not isinstance(blk, expected_cls):
[   27s]             raise ValueError('{0}: expecting class {1}'.format(
[   27s]                 arg_name, expected_cls.__name__))
[   27s]     
[   27s]         if self.__le and blk.__hdr_fmt__[0] == '>':
[   27s]             raise ValueError('{0}: expecting class {1}LE on a little-endian system'.format(
[   27s]                 arg_name, expected_cls.__name__))
[   27s]     
[   27s]         if not self.__le and blk.__hdr_fmt__[0] == '<':
[   27s] >           raise ValueError('{0}: expecting class {1} on a big-endian system'.format(
[   27s]                 arg_name, expected_cls.__name__.replace('LE', '')))
[   27s] E           ValueError: shb: expecting class SectionHeaderBlock on a big-endian system
[   27s] 
[   27s] dpkt/pcapng.py:343: ValueError
[   27s] =============================== warnings summary ===============================
[   27s] dpkt/decorators.py::TestDeprecatedDecorator::test_deprecated_decorator
[   27s]   /home/abuild/rpmbuild/BUILD/dpkt-1.9.2/dpkt/decorators.py:58: DeprecationWarning: Call to deprecated method deprecated_decorator
[   27s]     self.deprecated_decorator()
[   27s] 
[   27s] -- Docs: https://docs.pytest.org/en/stable/warnings.html
[   27s] =========================== short test summary info ============================
[   27s] FAILED dpkt/__init__.py::test_utils - assert 54720 == 51150
[   27s] FAILED dpkt/dpkt.py::test_utils - assert 54720 == 51150
[   27s] FAILED dpkt/ieee80211.py::test_80211_beacon - assert 0 == 1
[   27s] FAILED dpkt/ieee80211.py::test_compressed_block_ack - dpkt.dpkt.UnpackError: ...
[   27s] FAILED dpkt/ieee80211.py::test_action_block_ack_request - assert 528 == 4098
[   27s] FAILED dpkt/ieee80211.py::test_action_block_ack_response - assert 34835 == 5000
[   27s] FAILED dpkt/pcapng.py::test_custom_read_write - ValueError: shb: expecting cl...
[   27s] =================== 7 failed, 178 passed, 1 warning in 1.08s ===================
mcepl commented 3 years ago

Patch skips failing tests.

obormot commented 3 years ago

@mcepl I've got a branch up with fixes for big endian, producing no test failures on a PowerPC emulator. I couldn't get the s390x emulator working fast enough. Feel free to give it a try! https://github.com/kbandla/dpkt/tree/fix-big-endian-2

obormot commented 3 years ago

UPD: thanks to @brifordwylie 's speedy review this has been merged to master

mcepl commented 2 years ago

I am afraid that even with 1.9.7.2, we are still not there:

[  261s] =================================== FAILURES ===================================
[  261s] _____________________________ test_ethernet_unpack _____________________________
[  261s] 
[  261s]     def test_ethernet_unpack():
[  261s]         buf = b'\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x08\x00'
[  261s]         hdr = b'\x00\x02\x00\x02'
[  261s]     
[  261s]         lo = Loopback(hdr + buf)
[  261s] >       assert lo.family == 33554944
[  261s] E       assert 131074 == 33554944
[  261s] E         +131074
[  261s] E         -33554944
[  261s] 
[  261s] dpkt/loopback.py:46: AssertionError
[  261s] _______________________________ test_ip6_unpack ________________________________
[  261s] 
[  261s]     def test_ip6_unpack():
[  261s]         import struct
[  261s]         buf = (b'\x60\x00\x00\x00\x00\x14\x06\x38\x26\x07\xf8\xb0\x40\x0c\x0c\x03\x00\x00\x00\x00\x00\x00'
[  261s]                b'\x00\x1a\x20\x01\x04\x70\xe5\xbf\xde\xad\x49\x57\x21\x74\xe8\x2c\x48\x87')
[  261s]         hdr_suffix = b'\x00' * 3
[  261s]     
[  261s]         for family in (24, 28, 30):
[  261s]             hdr = struct.pack('B', family) + hdr_suffix
[  261s]     
[  261s]             lo = Loopback(hdr + buf)
[  261s] >           assert lo.family == family
[  261s] E           assert 402653184 == 24
[  261s] E             +402653184
[  261s] E             -24
[  261s] 
[  261s] dpkt/loopback.py:73: AssertionError
[  261s] _____________________________ test_writepkt_epb_ts _____________________________
[  261s] 
[  261s] args = (), kwargs = {}, fobj = <_io.BytesIO object at 0x3ff9d6b4af0>
[  261s] 
[  261s]     def wrapper(*args, **kwargs):
[  261s]         fobj = BytesIO()
[  261s]         f.__globals__['fobj'] = fobj
[  261s] >       ret = f(*args, **kwargs)
[  261s] 
[  261s] dpkt/pcapng.py:898: 
[  261s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[  261s] dpkt/pcapng.py:1194: in test_writepkt_epb_ts
[  261s]     writer = Writer(fobj, shb=shb, idb=idb)  # noqa
[  261s] dpkt/pcapng.py:371: in __init__
[  261s]     self._validate_block('shb', shb, SectionHeaderBlock)
[  261s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[  261s] 
[  261s] self = <dpkt.pcapng.Writer object at 0x3ff9d8a0438>, arg_name = 'shb'
[  261s] blk = SectionHeaderBlockLE(opts=[PcapngOptionLE(code=3, data=b'64-bit Windows 8.1, build 9600'), PcapngOptionLE(code=4, data=b'Dumpcap 1.12.7 (v1.12.7-0-g7fc8978 from master-1.12)'), PcapngOptionLE(opt_endofopt)])
[  261s] expected_cls = <class 'dpkt.pcapng.SectionHeaderBlock'>
[  261s] 
[  261s]     def _validate_block(self, arg_name, blk, expected_cls):
[  261s]         """Check a user-defined block for correct type and endianness"""
[  261s]         if not isinstance(blk, expected_cls):
[  261s]             raise ValueError('{0}: expecting class {1}'.format(
[  261s]                 arg_name, expected_cls.__name__))
[  261s]     
[  261s]         if self.__le and blk.__hdr_fmt__[0] == '>':
[  261s]             raise ValueError('{0}: expecting class {1}LE on a little-endian system'.format(
[  261s]                 arg_name, expected_cls.__name__))
[  261s]     
[  261s]         if not self.__le and blk.__hdr_fmt__[0] == '<':
[  261s]             raise ValueError('{0}: expecting class {1} on a big-endian system'.format(
[  261s] >               arg_name, expected_cls.__name__.replace('LE', '')))
[  261s] E           ValueError: shb: expecting class SectionHeaderBlock on a big-endian system
[  261s] 
[  261s] dpkt/pcapng.py:405: ValueError
[  261s] ____________________________ test_custom_read_write ____________________________
[  261s] 
[  261s]     def test_custom_read_write():
[  261s]         """Test a full pcapng file with 1 ICMP packet"""
[  261s]         buf = define_testdata().valid_pcapng
[  261s]         fobj = BytesIO(buf)
[  261s]     
[  261s]         # test reading
[  261s]         reader = Reader(fobj)
[  261s]         assert reader.snaplen == 0x40000
[  261s]         assert reader.datalink() == DLT_EN10MB
[  261s]     
[  261s]         assert reader.idb.opts[0].data.decode('utf-8') == '\\Device\\NPF_{3BBF21A7-91AE-4DDB-AB2C-C782999C22D5}'
[  261s]         assert reader.idb.opts[2].data.decode('utf-8') == '64-bit Windows 8.1, build 9600'
[  261s]     
[  261s]         ts, buf1 = next(iter(reader))
[  261s]         assert ts == 1442984653.2108380
[  261s]         assert len(buf1) == 74
[  261s]     
[  261s]         assert buf1.startswith(b'\x08\x00\x27\x96')
[  261s]         assert buf1.endswith(b'FGHI')
[  261s]         fobj.close()
[  261s]     
[  261s]         # test pcapng customized writing
[  261s]         if sys.byteorder == 'little':
[  261s]             shb, idb, epb = define_testdata().shb_idb_epb_le
[  261s]         else:
[  261s]             shb, idb, epb = define_testdata().shb_idb_epb_be
[  261s]     
[  261s]         fobj = BytesIO()
[  261s]         writer = Writer(fobj, shb=shb, idb=idb)
[  261s]         writer.writepkt(epb, ts=1442984653.210838)
[  261s] >       assert fobj.getvalue() == buf
[  261s] E       assert b"\n\r\r\n\x0...0\x00\x00\x84" == b"\n\r\r\n|\x...4\x00\x00\x00"
[  261s] E         At index 4 diff: b'\x00' != b'|'
[  261s] E         Full diff:
[  261s] E           (
[  261s] E         -  b'\n\r\r\n|\x00\x00\x00M<+\x1a\x01\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff'
[  261s] E         ?            -            ^^^        ----
[  261s] E         +  b'\n\r\r\n\x00\x00\x00|\x1a+<M\x00\x01\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff'
[  261s] E         ?                        ^    +++++++...
[  261s] E         
[  261s] E         ...Full output truncated (36 lines hidden), use '-vv' to show
[  261s] 
[  261s] dpkt/pcapng.py:1149: AssertionError
[  261s] ____________________________ test_multi_idb_writer _____________________________
[  261s] 
[  261s]     def test_multi_idb_writer():
[  261s]         """Test writing multiple interface description blocks into pcapng and read it"""
[  261s]         fobj = BytesIO()
[  261s]         shb, idb, epb = define_testdata().shb_idb_epb_le
[  261s]     
[  261s] >       writer = Writer(fobj, shb=shb, idb=[idb, idb])
[  261s] 
[  261s] dpkt/pcapng.py:1168: 
[  261s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[  261s] dpkt/pcapng.py:371: in __init__
[  261s]     self._validate_block('shb', shb, SectionHeaderBlock)
[  261s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[  261s] 
[  261s] self = <dpkt.pcapng.Writer object at 0x3ff9d930240>, arg_name = 'shb'
[  261s] blk = SectionHeaderBlockLE(opts=[PcapngOptionLE(code=3, data=b'64-bit Windows 8.1, build 9600'), PcapngOptionLE(code=4, data=b'Dumpcap 1.12.7 (v1.12.7-0-g7fc8978 from master-1.12)'), PcapngOptionLE(opt_endofopt)])
[  261s] expected_cls = <class 'dpkt.pcapng.SectionHeaderBlock'>
[  261s] 
[  261s]     def _validate_block(self, arg_name, blk, expected_cls):
[  261s]         """Check a user-defined block for correct type and endianness"""
[  261s]         if not isinstance(blk, expected_cls):
[  261s]             raise ValueError('{0}: expecting class {1}'.format(
[  261s]                 arg_name, expected_cls.__name__))
[  261s]     
[  261s]         if self.__le and blk.__hdr_fmt__[0] == '>':
[  261s]             raise ValueError('{0}: expecting class {1}LE on a little-endian system'.format(
[  261s]                 arg_name, expected_cls.__name__))
[  261s]     
[  261s]         if not self.__le and blk.__hdr_fmt__[0] == '<':
[  261s]             raise ValueError('{0}: expecting class {1} on a big-endian system'.format(
[  261s] >               arg_name, expected_cls.__name__.replace('LE', '')))
[  261s] E           ValueError: shb: expecting class SectionHeaderBlock on a big-endian system
[  261s] 
[  261s] dpkt/pcapng.py:405: ValueError
[  261s] =============================== warnings summary ===============================
[  261s] dpkt/ip.py::test_property_setters
[  261s]   /home/abuild/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ip.py:128: UserWarning: IP.off is deprecated
[  261s]     deprecation_warning("IP.off is deprecated")
[  261s] 
[  261s] dpkt/ip.py::test_property_setters
[  261s]   /home/abuild/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ip.py:123: UserWarning: IP.off is deprecated
[  261s]     deprecation_warning("IP.off is deprecated")
[  262s] 
[  262s] dpkt/ssl.py::TestServerHello::test_ciphersuite
[  262s]   /home/abuild/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ssl.py:350: UserWarning: TLSServerHello.cipher_suite is deprecated and renamed to .ciphersuite
[  262s]     deprecation_warning("TLSServerHello.cipher_suite is deprecated and renamed to .ciphersuite")
[  262s] 
[  262s] dpkt/ssl.py::TestServerHello::test_compression_method
[  262s]   /home/abuild/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ssl.py:355: UserWarning: TLSServerHello.compression is deprecated and renamed to .compression_method
[  262s]     deprecation_warning("TLSServerHello.compression is deprecated and renamed to .compression_method")
[  262s] 
[  262s] -- Docs: https://docs.pytest.org/en/stable/warnings.html
[  262s] =========================== short test summary info ============================
[  262s] FAILED dpkt/loopback.py::test_ethernet_unpack - assert 131074 == 33554944
[  262s] FAILED dpkt/loopback.py::test_ip6_unpack - assert 402653184 == 24
[  262s] FAILED dpkt/pcapng.py::test_writepkt_epb_ts - ValueError: shb: expecting clas...
[  262s] FAILED dpkt/pcapng.py::test_custom_read_write - assert b"\n\r\r\n\x0...0\x00\...
[  262s] FAILED dpkt/pcapng.py::test_multi_idb_writer - ValueError: shb: expecting cla...
[  262s] ================== 5 failed, 420 passed, 4 warnings in 26.59s ==================

That's with s390x and ppc64, all tests pass with x86_64 and similar (including ppc64le).

Complete build log

obormot commented 2 years ago

These are likely new issues since all earlier issues were fixed. I'll just reopen this ticket.

mcepl commented 2 years ago

I used to be a maintainer of PSPP in Fedora and I have learned one thing: endianness is a bitch; forgive me my French. When we started to build on SPARC we had by far more bugs from LE/BE than from anything else combined.

obormot commented 2 years ago

@crocogorical I see you added unit tests to https://github.com/kbandla/dpkt/blob/master/dpkt/loopback.py that are partially failing on big endian. Can you take a look? I have no clue what specific protocol this decoder is processing. There are no references to RFC etc.

obormot commented 2 years ago

FYI https://github.com/kbandla/dpkt/pull/615

obormot commented 2 years ago

Now fixed in master branch, closing.