Illegal backwards jump when decoding OpenSea function calls

[onesuper]

Hi @esaulpaugh,

Thanks a lot for having done such a great job.

We use headlong as our core decoding facilitates at our open-source analytics library: https://github.com/datawaves-xyz/blockchain-spark. It is super fast, well maintained, and suits our use case.

Recently, we found some decoding errors from our server log. It almost failed on half of OpenSea's 'atomicMatch' function calls (address: 0x7f268357a8c2552623316e2562d90e642bb538e5). I try to debug the program but hard to fix it by my self. So I am asking you for help.

Here's the stack trace:

java.lang.IllegalArgumentException: illegal backwards jump: (4+2848=2852)<2884
    at com.esaulpaugh.headlong.abi.TupleType.decodeObjects(TupleType.java:306)
    at com.esaulpaugh.headlong.abi.TupleType.decode(TupleType.java:193)
    at com.esaulpaugh.headlong.abi.Function.decodeCall(Function.java:233)
(...)

I retrieve the data from our server. I decode it with Web3.py on my laptop then it works. The following code snippet is for reproducing the problem locally.

Function f = Function.fromJson("{\"constant\": false, \"inputs\": [{\"name\": \"addrs\", \"type\": \"address[14]\"}, {\"name\": \"uints\", \"type\": \"uint256[18]\"}, {\"name\": \"feeMethodsSidesKindsHowToCalls\", \"type\": \"uint8[8]\"}, {\"name\": \"calldataBuy\", \"type\": \"bytes\"}, {\"name\": \"calldataSell\", \"type\": \"bytes\"}, {\"name\": \"replacementPatternBuy\", \"type\": \"bytes\"}, {\"name\": \"replacementPatternSell\", \"type\": \"bytes\"}, {\"name\": \"staticExtradataBuy\", \"type\": \"bytes\"}, {\"name\": \"staticExtradataSell\", \"type\": \"bytes\"}, {\"name\": \"vs\", \"type\": \"uint8[2]\"}, {\"name\": \"rssMetadata\", \"type\": \"bytes32[5]\"}], \"name\": \"atomicMatch_\", \"outputs\": [], \"payable\": true, \"stateMutability\": \"payable\", \"type\": \"function\"}");
Tuple inputTuple = f.decodeCall(FastHex.decode("ab834bab0000000000000000000000007f268357a8c2552623316e2562d90e642bb538e5000000000000000000000000267011df2647c0a465a7d0aacb8257483db6b6c3000000000000000000000000f5962dd1cfb95bd15a598b721091b373fcf1e3d10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000baf2127b49fc93cbca6269fade0f7f31df4c88a7000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f268357a8c2552623316e2562d90e642bb538e5000000000000000000000000f5962dd1cfb95bd15a598b721091b373fcf1e3d100000000000000000000000000000000000000000000000000000000000000000000000000000000000000005b3256965e7c3cf26e11fcaf296dfc8807c01073000000000000000000000000baf2127b49fc93cbca6269fade0f7f31df4c88a70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002ee0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000429d069189e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006235fb0800000000000000000000000000000000000000000000000000000000000000006d30435dda06a6f4136149c29818e18c3f3843728e80259fa66f1201fbdb91f000000000000000000000000000000000000000000000000000000000000002ee0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000429d069189e00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006232b917000000000000000000000000000000000000000000000000000000006246314446231f8dcf1e298e1c8579f603c6fa74fa51f0f886c4ae376377eb08467f3c290000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000006a000000000000000000000000000000000000000000000000000000000000007c000000000000000000000000000000000000000000000000000000000000008e00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000b200000000000000000000000000000000000000000000000000000000000000b20000000000000000000000000000000000000000000000000000000000000001c000000000000000000000000000000000000000000000000000000000000001c7596ed295fccb24c5008d11d537120b1b220dbe835bfd0ae15373d9f8e6a947c46c70ad794e802c20c451e530c4cf6962696174a2169e03f2f2a64000f93ea2a7596ed295fccb24c5008d11d537120b1b220dbe835bfd0ae15373d9f8e6a947c46c70ad794e802c20c451e530c4cf6962696174a2169e03f2f2a64000f93ea2a000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4fb16a5950000000000000000000000000000000000000000000000000000000000000000000000000000000000000000267011df2647c0a465a7d0aacb8257483db6b6c3000000000000000000000000920a8d9e9f0defd6f86e4388a5503b04cac83b5700000000000000000000000000000000000000000000000000000000000031be000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4fb16a595000000000000000000000000f5962dd1cfb95bd15a598b721091b373fcf1e3d10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000920a8d9e9f0defd6f86e4388a5503b04cac83b5700000000000000000000000000000000000000000000000000000000000031be000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e400000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4000000000000000000000000000000000000000000000000000000000000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"));

To get you to the site faster, the error happens when parsing the 8th element of the tuple:

Thanks again!

Yichao

[esaulpaugh]

A backwards jump (where the destination offset is less than the current offset) would indicate that bytes that have already been decoded should be decoded again, and for a different purpose. As a security precaution and sanity check, I have not been allowing this.

Could you contact the creator of the data to determine if they really intend to do this and if so, why?

[esaulpaugh]

Specifically, they are reusing the same offset value (2848) for two distinct items (index 7 and index 8). I assume they are doing it intentionally (to save gas?), but it is best to be sure.

[onesuper]

I am glad you reply. I will contact the author to figure out what's the intention.

Should we consider relaxing the restriction only if the outcome is right? But I'm not sure whether it is a good choice from the API perspective. Maybe this case (reusing the same offset) is not very common.

We have some customers who are facing this issue. It begs the question: How can I patch the code to avoid the error? Maybe I can craft a patch to work around the problem temporarily.

[esaulpaugh]

I added a commit which should allow any number of elements to share the same offset, but which will not allow backwards jumps in other circumstances: https://github.com/esaulpaugh/headlong/commit/e042a122499dec0a040f64cbd3780112a4e17e21

If you are able to test this change on the full dataset, please do.

In any case, I will most likely release a new version with this change included soon.

[onesuper]

Cool! I will test it right now (I will come back if I get testing results). The patch looks great to me.

[onesuper]

Problem solved. Many thanks!

The result of decoding OpenSea Contract from datawaves