search

Consensys / linea-tracer

Part of the Linea stack responsible for extracting data from the execution of an EVM client in order to construct large matrices called execution traces.
https://linea.build
Other
35 stars 22 forks source link

non-determinism - mainnet v0.6.0-rc8 (txndata error) #1316

Closed gusiri closed 1 month ago

gusiri commented 1 month ago

Summary:

For those ranges, traces generated by mainnet-1 (10.40.42.125) always fail.

Range mainnet-1 (10.40.42.125) mainnet-2 (10.40.43.192)
3424829-3424864 Fail Pass
2460617-2460668 Fail Pass
2519403-2519438 Fail Pass
3004582-3004619 Fail Pass
4477086-4477226 Fail Pass
2435888-2435925 Fail Pass
2976387-2976433 Fail Pass
2503100-2503131 Fail Pass
3013150-3013187 Fail Pass
5070155-5070199 Fail Pass
and more ... Fail Pass

Error message:

FAILED
err = Several errors were encountered:
     * <nil>
failed (vanish txndata.setting-refund-effective... the global constraint check failed at row ...

When I split the range into single blocks and generate them on the problematic besu-mainnet-1 (10.40.42.125), they all passed. I tested this for 3424829-3424864 and 5070155-5070199. I assume this applies to all the ranges above.

Test environment

we have only 1 problematic node, let's say it's mainnet-1

Version info

linea-besu: 915fcb01 linea-tracer: v0.6.0-rc8 (54c3499)

linea-besu: 80c7e4d linea-tracer: v0.6.0-rc8 (54c3499)

Instance types

Kernel Info

EBS configuration

To make sure there's no CPU IO wait

Non-deterministic Traces

Type 1

3424829-3424864

----
## Type 2
- [ ] txndata
`v0.6.0-rc1`: `lookup` errors (https://github.com/Consensys/linea-tracer/issues/1259).
`v0.6.0-rc5` no errors 
`v0.6.0-rc7` `txndata` errors
`v0.6.0-rc8` `txndata` errors

2460617-2460668 2519403-2519438 3004582-3004619 4477086-4477226 2435888-2435925 2976387-2976433 3424829-3424864 2503100-2503131 3013150-3013187


----
## Type 3
- [ ] txndata 
`v0.6.0-rc8`: `txndata` error
Looks like new ranges.  
(lt file: https://drive.google.com/drive/folders/1zPwMYK3BOMbDyB8epSYLPTFUqQlF21-9?usp=drive_link)

5070155-5070199 2622969-2623015 2643006-2643062 2721798-2721890 2815112-2815163 2890024-2890052 3068429-3068462 3113794-3113813 3126838-3126852 3222386-3222400 3250855-3250863 3283554-3283560 3286982-3286989 3349132-3349148 3427261-3427297 3624990-3625017 3739694-3739755 3768979-3769044 4093510-4093574 4108923-4108979 4179262-4179347 4437150-4437235 4459044-4459124 4846898-4846981 4860879-4860916 4964388-4964507 4977689-4977733 5126607-5126648

OlivierBBB commented 1 month ago

Upon running 3424829-3424864 all transactions get reverted it seems:

SLF4J(W): Class path contains multiple SLF4J providers.
SLF4J(W): Found provider [ch.qos.logback.classic.spi.LogbackServiceProvider@5b3f3ba0]
SLF4J(W): Found provider [org.apache.logging.slf4j.SLF4JServiceProvider@ddf20fd]
SLF4J(W): See https://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J(I): Actual provider is of type [ch.qos.logback.classic.spi.LogbackServiceProvider@5b3f3ba0]
20:49:33.217 [Test worker] INFO  - Bonsai flat db mode found FULL
20:49:33.223 [Test worker] INFO  - Bonsai db mode with code stored using code hash enabled = true
20:49:34.140 [Test worker] INFO  - Protocol schedule created with milestones: [London:0]
20:49:35.331 [Test worker] DEBUG - Transaction 0x130e8d3d21ec9420a0d7f327a3bbe7dcbfbb64e490d981c1fc528d1a265c22ca reverted: 0x
20:49:36.193 [Test worker] DEBUG - Transaction 0x49bf0cbbea7b0254a22be419ba381732eaa938c6f2cd737905f016b075ea615f reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:37.092 [Test worker] DEBUG - Transaction 0x4e46b6a3eecebccb92c201b27be3ff6e3db5764ed1f318f64db95ff775a0ed37 reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:37.113 [Test worker] DEBUG - Transaction 0x2d00ed27ccde2bdab64078a17c9f3b08f473c420d89228dea744a1773af39fd8 reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:37.131 [Test worker] DEBUG - Transaction 0x7e7dffc2bc3965488baa350872c3c7331201a3e7d4441302af650693272f2816 reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:37.149 [Test worker] DEBUG - Transaction 0x391624de6e318496d0862b43edb39faf45c345274c3e301d430d36febbd178ea reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:39.858 [Test worker] DEBUG - Transaction 0x26c7eeda1a78f86597092cff2d98f7f5360cf10b3b8b954c8cbbf79292a94fe1 reverted: 0x
20:49:41.104 [Test worker] DEBUG - Transaction 0xf0b24325c4e344fbe11417824fafc715f4afde418039da726296b759624cdaf4 reverted: 0x
20:49:41.617 [Test worker] DEBUG - Transaction 0xdd92beed468b2fa8020a44e64c8a39a93f7813760cd47eadb639158ac7d20adc reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001b52657475726e20616d6f756e74206973206e6f7420656e6f7567680000000000
20:49:42.369 [Test worker] DEBUG - Transaction 0x3dc47fb1135c014754af85d8c2fb6257cfeb94a3dd6baebeda21128ab4246ca9 reverted: 0x
20:49:44.193 [Test worker] DEBUG - Transaction 0xf67df0e13500c6b582f011027c5ef0448881da3a3ab3ec0af7e7a8dafa68fea5 reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:44.691 [Test worker] DEBUG - Transaction 0x1a01304b263ef5fca12b88ea333ffbd88f06b67280e323a29808e206cf1ac8db reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:44.708 [Test worker] DEBUG - Transaction 0xbd822208e45fee24c6625f8232e0b6a9163776790aa66d3aa6ac5d18db6a74bc reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:44.810 [Test worker] DEBUG - Transaction 0x127326bac039c7845188c59ec41ff65da48f9946beb29c61e31fdd8d9a1f0350 reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:45.404 [Test worker] DEBUG - Transaction 0x150901501d4fb6e72239613732d96c84d7c1bed2e5cb8b319b77a540cdb4251c reverted: 0x
20:49:45.512 [Test worker] DEBUG - Transaction 0x9235cc04afbbcb9b06236ae85121c6d7a89342b9d0ba58e3839b2107d9e108ff reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:45.630 [Test worker] DEBUG - Transaction 0xaee1fab10170272b6e5f3dca8f9af19f34ef0887d58c0d870f0612b151a149b3 reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001a494e53554646494349454e545f4f55545055545f414d4f554e54000000000000
20:49:45.687 [Test worker] DEBUG - Transaction 0x7c4bccd2a4ce69e040a55213e9ec05bdd6ada2f7f771e5ffa26f29e2bd4da5c7 reverted: 0x
20:49:49.421 [Test worker] DEBUG - Transaction 0x5312a2b796ec0e1e54bf1e5db13ecf311f1dcc32ce0208172cd623d96b2b0786 reverted: 0x08c379a0000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000504f70656e4f6365616e2065787465726e616c2063616c6c206661696c65643a204572726f72284f70656e4f6365616e2065787465726e616c2063616c6c206661696c65643a20556e6b6e6f776e28292900000000000000000000000000000000
20:49:49.735 [Test worker] DEBUG - Transaction 0x43e8d3b031da7275bdbd3bc6fac52c22ace5b0137d2d36960d3c30039f46c7fc reverted: 0x08c379a000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000010496e73756666696369656e742066656500000000000000000000000000000000
20:49:50.235 [Test worker] DEBUG - Transaction 0x098a51a90147211eed6ddee284ea4c08a475a9060326485cbcf2f1441eebd087 reverted: 0x08c379a000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000001ddfd51d000000000000000000000000176211869ca2b568f2a7d4ee941e073a0000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000001ddfd51d000000000000
20:49:52.015 [Test worker] DEBUG - Transaction 0xc96a26af7fbf0d27652a29903f7008bb57584287b44afe3f4e37a930bf238216 reverted: 0x
20:49:54.364 [Test worker] DEBUG - Transaction 0x263ff7718402abb0b1b357074aa5ca5aac3217928afdc162ba75d4b89d8c67e8 reverted: 0x08c379a000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000023426173655631526f757465723a20494e53554646494349454e545f415f414d4f554e540000000000000000000000000000000000000000000000000000000000
20:49:56.640 [Test worker] DEBUG - Transaction 0xd12b70fcc9a1a214a43a36613a163358082f68c91572697bd88746424a33742e reverted: 0x08c379a000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000014546f6b656e20616c7265616479206d696e746564000000000000000000000000
20:49:58.018 [Test worker] DEBUG - Transaction 0xb2be92ab48c869037e2628d57f502a48f54524bb1250159b5100d543aff93e85 reverted: 0x
20:50:02.471 [Test worker] DEBUG - Transaction 0x614534af54072710785cb6af41e6f8cb10c0f270f5d410d2094a9975ffffe490 reverted: 0x
20:50:03.180 [Test worker] DEBUG - Transaction 0x9d9d32790f108d156b1f8f636425345a679abef31850a9fa69a985957abf54ed reverted: 0x08c379a0000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000205361666545524332303a206c6f772d6c6576656c2063616c6c206661696c6564
20:50:05.100 [Test worker] DEBUG - Transaction 0xc9793aca17b9a704f6c8add58c525399bf165c1580958a6dd388245814b49151 reverted: 0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000001d47656e6572696353696756657269666965723a2077726f6e6720736967000000
20:50:05.685 [Test worker] DEBUG - Transaction 0xebeb95fe1d2e3806777366d1dfe1953900bb17b05f8e57822b359f1af2fc7427 reverted: 0x08c379a0000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000029f2b3b8f416c6000000000000000000000000e5d7c2a44ffddf6b295a15c148167daa00000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000029f2b3b8f416c6000000000000

Also I don't know exactly how to interpret the string after the transaction hash (?) but if this is the call data then all transactions are calling the function with function identifier 0x08c379a0 which stands for

image
OlivierBBB commented 1 month ago

Yet the test worked regardless

image
DavePearce commented 1 month ago

@OlivierBBB

Also I don't know exactly how to interpret the string after the transaction hash

Its not the CALLDATA but the bytes returned in memory as part of the REVERT. As discussed, these reverts are correct ... they should revert. However, not all transactions reverted. We don't see those which simply pass as expected.

If there was any discrepancy between the replay and mainnet ... the result checker would tell us. That's what its for :)