Decoding data of type byte[] array results in an an empty element at the end

[JanRuettinger]

What was wrong?

Can't decode type 'bytes[]' properly using the eth_abi.decode function. Instead of one, two elements of type byte are returned whereas the second element is always empty.

Code that produced the error

# pip install eth_abi hexbytes

from eth_abi import decode
from hexbytes import HexBytes

# Link to tx: https://etherscan.io/vmtrace?txhash=0x862c28d81477ea3cb37ac2a3df5f7efb451b1339fb0948c6b5d99992a94de05d&type=parity
output_raw = "0x0000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000006aeaeff8cc50ece0000000000000000000000000000000000000000000000000000000000000000"
output_hex_bytes = HexBytes(output_raw)
# output_hex_bytes: b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x06\xae\xae\xff\x8c\xc5\x0e\xce\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
to_decode_types = ['bytes[]']

decoded = decode(to_decode_types, output_hex_bytes)
print(decoded)
# result: ((b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x06\xae\xae\xff\x8c\xc5\x0e\xce', b''),)
# expected: ((b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x06\xae\xae\xff\x8c\xc5\x0e\xce'),)

Expected Result

((b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x06\xae\xae\xff\x8c\xc5\x0e\xce'),)

Environment

# run this:
$ python -m eth_utils

# then copy the output here:
Python version:
3.8.1 (v3.8.1:1b293b6006, Dec 18 2019, 14:08:53) 
[Clang 6.0 (clang-600.0.57)]

Operating System: macOS-10.16-x86_64-i386-64bit

pip freeze result:
anyio==3.6.1
appnope==0.1.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.0.8
attrs==22.1.0
Babel==2.10.3
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.1
certifi==2022.9.14
cffi==1.15.1
charset-normalizer==2.1.1
cytoolz==0.12.0
debugpy==1.6.3
decorator==5.1.1
defusedxml==0.7.1
entrypoints==0.4
eth-abi==3.0.1
eth-hash==0.3.3
eth-typing==3.2.0
eth-utils==2.0.0
executing==1.0.0
fastjsonschema==2.16.1
hexbytes==0.3.0
idna==3.4
importlib-metadata==4.12.0
importlib-resources==5.9.0
ipykernel==6.15.3
ipython==8.5.0
ipython-genutils==0.2.0
jedi==0.18.1
Jinja2==3.1.2
json5==0.9.10
jsonschema==4.16.0
jupyter-core==4.11.1
jupyter-server==1.18.1
jupyter_client==7.3.5
jupyterlab==3.4.7
jupyterlab-pygments==0.2.2
jupyterlab_server==2.15.1
lxml==4.9.1
MarkupSafe==2.1.1
matplotlib-inline==0.1.6
mistune==2.0.4
nbclassic==0.4.3
nbclient==0.6.8
nbconvert==7.0.0
nbformat==5.5.0
nest-asyncio==1.5.5
notebook==6.4.12
notebook-shim==0.1.0
packaging==21.3
pandocfilters==1.5.0
parsimonious==0.8.1
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
pkgutil_resolve_name==1.3.10
prometheus-client==0.14.1
prompt-toolkit==3.0.31
psutil==5.9.2
ptyprocess==0.7.0
pure-eval==0.2.2
pycparser==2.21
Pygments==2.13.0
pyparsing==3.0.9
pyrsistent==0.18.1
python-dateutil==2.8.2
pytz==2022.2.1
pyzmq==24.0.0
requests==2.28.1
Send2Trash==1.8.0
six==1.16.0
sniffio==1.3.0
soupsieve==2.3.2.post1
stack-data==0.5.0
terminado==0.15.0
tinycss2==1.1.1
tomli==2.0.1
toolz==0.12.0
tornado==6.2
traitlets==5.4.0
urllib3==1.26.12
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.4.1
zipp==3.8.1

[fselmo]

I apologize for the late response here @JanRuettinger but it looks like this was an outdated version of eth-abi. I'm actually not sure that bytes32[] is the right type for this call. I believe these are integers. [bad assumption, see next comments for answer]

>>> from eth_abi import abi
>>> from eth_utils import to_bytes

>>> raw_output_hex = "0x0000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000006aeaeff8cc50ece0000000000000000000000000000000000000000000000000000000000000000"
>>> output_bytes = to_bytes(hexstr=raw_output_hex)

>>> abi.decode(["uint256[]"], output_bytes)
((64, 128),)

>>> # with bytes32[]
>>> abi.decode(["bytes[]"], output_bytes)
((b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00@',
  b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80'),)

I'm going to close this as I can't reproduce with the latest version. But please ask to re-open if there are still any issues.

[fselmo]

Oh I see. That is my mistake for using bytes32[] and not bytes[]. I can reproduce what you have there but are you sure this is the type you need? Are you sure these aren't integers to decode to? [see next comment]

[fselmo]

I did some snooping and looked for bytes[] return types on the methods in that contract and I do see that it was a call to multicall(uint256,bytes[]) which matches the function signature on that call.

However, what eth-abi returns is exactly what Solidity returns, so I don't see a bug here.

function decode(bytes memory _bytes) external pure returns (bytes[] memory) {
    bytes[] memory _decoded = abi.decode(_bytes, (bytes[]));
    return _decoded;
}

This is what I get as the return, notice the 0x (b"") at the end of the array

"bytes[]: 0x00000000000000000000000000000000000000000000000006aeaeff8cc50ece,0x"

---

edit: It looks like Action [13] in that trace link you sent (the very last trace) is actually the call with 0x. multicall(uint256,bytes[]) iterates on each value passed in to it and makes a call with each of them. That's why you only see one of the arguments in the next step... and you see the second argument in another step. Hope that clarifies what you were seeing.