decode_abi fails with Overflow error but decode_single does not

ankitchiplunkar commented 5 years ago

Version: '1.3.0'
Python: 3.6.7
OS: linux

What was wrong?

Cannot decode the first event of this transaction using decode_abi but can decode using decode_single: https://etherscan.io/tx/0xe331b086561bad5554503115728123d93b8cb64f89b71d94e90876127c773379#eventlog

Please include any of the following that are applicable:

Data of the event

>>> from eth_abi import decode_abi, decode_single
>>> from hexbytes import HexBytes
>>> variables = '0x000000000000000000000000f232cbab76d7e90911cd2573f1d8afe70d0c9f9b0000000000000000000000006ccc8faebd1dedd14831e80435c82930bec0db770000000000000000000000000000000000000000000000d3d6b463e32cc9400000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000'
>>> b_variables = HexBytes(variables)
>>> types = ['address', 'address', 'uint256', 'bytes']

Tried decoding using `decode_abi`

>>> decode_abi(types, b_variables)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/abi.py", line 96, in decode_abi
    return decoder(stream)
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_utils/functional.py", line 46, in inner
    return callback(fn(*args, **kwargs))
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 164, in decode
    yield decoder(stream)
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 136, in decode
    value = self.tail_decoder(stream)
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 118, in __call__
    return self.decode(stream)
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 186, in decode
    raw_data = self.read_data_from_stream(stream)
  File "/home/ankit/venv/lib/python3.6/site-packages/eth_abi/decoding.py", line 583, in read_data_from_stream
    data = stream.read(padded_length)
OverflowError: Python int too large to convert to C ssize_t

Can decode using `decode_single`

>>> decode_single('address', b_variables[:32])
'0xf232cbab76d7e90911cd2573f1d8afe70d0c9f9b'
>>> decode_single('address', b_variables[32:64])
'0x6ccc8faebd1dedd14831e80435c82930bec0db77'
>>> decode_single('uint256', b_variables[64:96])
3907734100000000000000
>>> decode_single('bytes', b_variables[96:])
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

How can it be fixed?

Don't know

davesque commented 5 years ago

The decode_abi function expects for the data stream to be tuple-encoded. That means that the value at b_variables[96:128] should be a byte offset (relative to the beginning of the tuple-encoded object) that points to the beginning of the encoding of the dynamic bytes value. The value at b_variables[96:128] is 0x20 which, when interpreted as an offset (unsigned integer), is 32. That points to b_variables[32]. It seems that you expect for b_variables[96] to be the beginning of the encoded bytes value. Log events are tuple-encoded so I think your data stream is malformed. I'm not sure where you obtained that encoded data.

However, this does reveal another issue which is that the lower-level stream call seems to expect that buffer sizes must be bounded in the range possible for ssize_t. I guess we don't currently handle this case properly. Practically speaking, we probably won't have to handle it for a long time or, at least, not until people start encoding bytes values with a length greater than that which can be represented by a 32 or 64 bit integer. That's a heckuva lot of data and I don't think we really need to worry about it for now.

davesque commented 5 years ago

I'm going to close this for now. Feel free to re-open the issue if you have any other questions or if you feel things haven't been resolved. Thanks for submitting your issue!

ankitchiplunkar commented 5 years ago

In a log there are topics and data fields which constitute the variables of an event log. If there are 3 topics and data in an event log. I would create the variables field as

# topic[0] is the event signature_hash 
variables = [to_hex(topic[1]) + to_hex(topic[2]) + to_hex(data)]

Since the decode_abi function expects byte offsets in the data_stream, my above approach would not work. How would you propose I should decode my event log using only types, topics and data of a log?

davesque commented 5 years ago

Where are the topic and data values coming from in your example?

ethereum / eth-abi