search

ChainSafe / forest

🌲 Rust Filecoin Node Implementation
https://forest.chainsafe.io
Apache License 2.0
637 stars 157 forks source link

Some error occurred in `compute_tipset_state_blocking` #4264

Closed elmattic closed 2 months ago

elmattic commented 6 months ago

Describe the bug

While generating test cases using many tipsets, I've found another potential issue in the EthGetBlockByNumber method.

To reproduce

  1. Add an EthGetBlockByNumber RcpTest using payload [String("0x1777be"), Bool(true)]
  2. Run ./forest-tool api compare --lotus /ip4/127.0.0.1/tcp/1234/http --forest /ip4/127.0.0.1/tcp/2345/http forest_snapshot_calibnet_2024-04-19_height_1539740.forest.car.zst --filter EthGetBlockByNumber
  3. See error in Forest daemon

Log output

Log Output ``` Forest tool: Request dump: RpcRequest { method_name: "Filecoin.EthGetBlockByNumber", params: Array [String("0x1777be"), Bool(true)], result_type: PhantomData, api_version: V1, timeout: 60s } Request params JSON: ["0x1777be",true] Lotus response: { "baseFeePerGas": "0x64", "difficulty": "0x0", "extraData": "0x", "gasLimit": "0x2540be400", "gasUsed": "0xfd68c18", "hash": "0x357ec3e64af2cda2dbe63af4db846f4ac8a97b7ab9f88c23bfd7308988b299fa", "logsBloom": "0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff", "miner": "0x0000000000000000000000000000000000000000", "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "nonce": "0x0000000000000000", "number": "0x1777be", "parentHash": "0xbad4fc050836cbb4482bdfebdb5652dbbc5e37ebfb80f7ece64192f7255833fd", "receiptsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347", "size": "0x0", "stateRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "timestamp": "0x662169f0", "totalDifficulty": "0x0", "transactions": [ { "accessList": [], "blockHash": "0x357ec3e64af2cda2dbe63af4db846f4ac8a97b7ab9f88c23bfd7308988b299fa", "blockNumber": "0x1777be", "chainId": "0x4cb2f", "from": "0xff000000000000000000000000000000000003f3", "gas": "0x323c3c1", "hash": "0x4910d2fb5fff84d8d4b47c25e505f56c945e5fa35ceb9181e64a4b926d704747", "input": "0x868e10c4000000000000000000000000000000000000000000000000000000000000000f00000000000000000000000000000000000000000000000000000000000000510000000000000000000000000000000000000000000000000000000000000060000000000000000000000000000000000000000000000000000000000000075a835903a9904400a4fa0381586086c01e5ed550263290b51684a99503bbf877f004f68725fefb405ff2583b54f2f1aefeea7c56612febd8d457b38f4f210f8a46646805fc30f0f52fbd869e8bee682bbf761a33553b2a56205cc2890c36e12833f9ad6d2692f0c0b961ddcd8eb68201586091b920d086caedb4efedf802412cfeac35e12d052c0c1dbd1fd26e9c872f28b77a4fb57bffbfcefecde0f4e51618462c00d50eb766d32df62e31c14a69355d12aa9294e8e10e3292a3cbff8d9a4c6823529f205348173a484e6a414429ccd7f881821a006918265830b09b0044f202abba1d1accb14406ede3ed9d50ccad4e7e03254c23f4a5e773c6aca1ab5a94c23de2903e45beafbdf05281820358c093d01e3ac799e7d253596275547f5267d0885d1086c6b69e2a936df074d2fc7b4f57bd17e936c3996e8977b0280ea87da3dfd204f4cf34b6416e99b420ce2336472d26ee89bc3488ee0cec4e47662adc418b2af084aa4b363b0acae34022dd9409c6f70e5fa746e6ccd4873f2e4699b5dd577324ee6b1331e80377a102372950ee1ae7ad07e3051831461fdad51dbd43875771b1af653e2d963cf07e2e24d8303fb1692403c58ce78cf358004bdf9b33da79afccd60d2375bc081cf037daefe483d82a5827000171a0e40220f80a8be9f108dbfd421517a4ce5397a21bc9e71c5a142d58959ce84ce9483087d82a5827000171a0e40220b593d29e9fb3c106f1dfbd0c50f5929cd7e6d374b4311ddc5e048244e6bd9451d82a5827000171a0e4022061f71a071c94eb4f72bd039641876cf2400701b32034d779b04358c0168da33e460006aa823f4c1a001777bcd82a5827000171a0e40220525384cf760c938df8095f7ad521285b7e0b1f0a2d1b899ca999bb8e307fa2c0d82a5827000171a0e4022040f581e65dd0affa8ea66604620477bcc01da33eb43c3b134537478605a30dc8d82a5827000171a0e40220a511127908777cdae5f7da7377dba637e2fb0bdb2f689414e901a093b3d6e834586102c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001a662169b4586102aa83d225457bb8f8922cb6ad1cda16046bc53538be75402c0a91138b84a194687a9ad3e10ae55fbefc90b9d945591fa1061af805ebb1f6dcf5b2bd3f51ecfcaa93c17518dc4d482646c0411508de9acda0d25a7fa9ad64f2368533322b557ce2004200645903a9904400a4fa0381586086c01e5ed550263290b51684a99503bbf877f004f68725fefb405ff2583b54f2f1aefeea7c56612febd8d457b38f4f210f8a46646805fc30f0f52fbd869e8bee682bbf761a33553b2a56205cc2890c36e12833f9ad6d2692f0c0b961ddcd8eb68201586091b920d086caedb4efedf802412cfeac35e12d052c0c1dbd1fd26e9c872f28b77a4fb57bffbfcefecde0f4e51618462c00d50eb766d32df62e31c14a69355d12aa9294e8e10e3292a3cbff8d9a4c6823529f205348173a484e6a414429ccd7f881821a006918265830b09b0044f202abba1d1accb14406ede3ed9d50ccad4e7e03254c23f4a5e773c6aca1ab5a94c23de2903e45beafbdf05281820358c0823d11e758566cb939f78f84ebad4dd728aef6ba44c385f39707cc5e7df303fa691d9eda847c04f6bfcdb9493ebf908eb9a76dfd1d68fedcadfa1636b19398a59951fb5bf282ecd63f971c40a877fda2ef0ed6fc2d3a502c728b61762f61d453020a191a7f52d4c23720afb97472edebe10874f3ba9c8cb903b21b6b961c7ad9f4b0e211f1fdf0d9ba8740a0ed0849a48d83a7fd3215ae546a1051e7e367ad0e0bebfd1ac901c9ce09a99cf6e0ec670cc2e7da1331fc7a1f1adf174fa3a998f683d82a5827000171a0e40220f80a8be9f108dbfd421517a4ce5397a21bc9e71c5a142d58959ce84ce9483087d82a5827000171a0e40220b593d29e9fb3c106f1dfbd0c50f5929cd7e6d374b4311ddc5e048244e6bd9451d82a5827000171a0e4022061f71a071c94eb4f72bd039641876cf2400701b32034d779b04358c0168da33e460006aa823f4c1a001777bcd82a5827000171a0e40220525384cf760c938df8095f7ad521285b7e0b1f0a2d1b899ca999bb8e307fa2c0d82a5827000171a0e4022040f581e65dd0affa8ea66604620477bcc01da33eb43c3b134537478605a30dc8d82a5827000171a0e40220a511127908777cdae5f7da7377dba637e2fb0bdb2f689414e901a093b3d6e834586102c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001a662169b4586102b4cfac12849af3e6363f0bc12053a5e3e77178db4b58911b39ac0daa34f3f26f8b70c9408f97283f7e12142ff4f6ef4407e9197bcfbc6dc6240985fc4baf4f7621f63d9cc101988b692e9652d575552e478e48912072c27240c41cc23c4345fa0042006440000000000000", "maxFeePerGas": "0x18d55", "maxPriorityFeePerGas": "0x18937", "nonce": "0x41", "r": "0x0", "s": "0x0", "to": "0xff0000000000000000000000000000000000fd24", "transactionIndex": "0x0", "type": "0x2", "v": "0x0", "value": "0x0" }, { "accessList": [], "blockHash": "0x357ec3e64af2cda2dbe63af4db846f4ac8a97b7ab9f88c23bfd7308988b299fa", "blockNumber": "0x1777be", "chainId": "0x4cb2f", "from": "0x3c71e88cc11204f981fbe577f8286f058e98a008", "gas": "0x131ad67c", "hash": "0xaa8f1d1e878b4403a6f9d639bf496afca20a832c34cfe7e3c79f5ed5ecf02fc9", "input": "0xd5b3d76d00000000000000000000000000000000000000000000000455c7c342c6ea40000000000000000000000000000000000000000000000000000000002e90edd000000000000000000000000000000000000000000000000000000000000007e900", "maxFeePerGas": "0x59682f78", "maxPriorityFeePerGas": "0x59682f00", "nonce": "0x1", "r": "0xd365983c397fff4dc93bd58087ce7493f3017c683d7648b03781916b57c03651", "s": "0x6f83138c5ca774982c9ff25c6acd3a59b9ff3f7e3e08b38dbc7dc6d42886d04f", "to": "0x7bf370666854a69ad103bda0f059746572ea3134", "transactionIndex": "0x1", "type": "0x2", "v": "0x1", "value": "0x0" }, { "accessList": [], "blockHash": "0x357ec3e64af2cda2dbe63af4db846f4ac8a97b7ab9f88c23bfd7308988b299fa", "blockNumber": "0x1777be", "chainId": "0x4cb2f", "from": "0x3bceb86a8326d9add82e906c20d4078547f13183", "gas": "0x5df482", "hash": "0x2dc91123b5e780e4cf8b5af77261e81f39fdf8c14970b690c23ebf72c7c55d48", "input": "0x", "maxFeePerGas": "0x2cd98932", "maxPriorityFeePerGas": "0x2cd98932", "nonce": "0x87", "r": "0x7c55ce0a6073c990c1950d82431974eace87409d96efa222c1e9749f5bd710c0", "s": "0x4169f35ea8011ef68bb9b7567cf8b004042975c5c5f483a965a24810228fafca", "to": "0x3dc390454d7c74c7078103feb57541c0f7c98b4f", "transactionIndex": "0x2", "type": "0x2", "v": "0x1", "value": "0x8ac7230489e80000" } ], "transactionsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000", "uncles": [] } | RPC Method | Forest | Lotus | |-----------------------------------|---------------------|-------| | Filecoin.EthGetBlockByNumber | InternalServerError | Valid | | Filecoin.EthGetBlockByNumber (20) | Valid | Valid | Error: Some tests failed Forest daemon: 2024-04-23T16:10:32.044347Z INFO forest_filecoin::chain_sync::tipset_syncer: Successfully synced tipset range: [1552074, 1552075] 2024-04-23T16:10:32.221228Z INFO forest_filecoin::chain_sync::tipset_syncer: Validating tipset: EPOCH = 1552075, N blocks = 1 2024-04-23T16:10:32.237870Z INFO forest_filecoin::chain::store::chain_store: New heaviest tipset! [bafy2bzacecwinmasofuetywj45pk2w57tua3rxuxatuxhth6m7x76sofdg2go, bafy2bzacealuya6wl3ppws4pdmcnyoieiw7yxlot3pgmx6bc7wfl6adcteuyy, bafy2bzacedlxtjngplyozwsbqfnlvqlciyygcyhxdbrjccvijsl7b4nia73js] (EPOCH = 1552075) 2024-04-23T16:10:32.237978Z INFO forest_filecoin::chain_sync::tipset_syncer: Successfully synced tipset range: [1552075, 1552075] 2024-04-23T16:10:42.997119Z ERROR compute_tipset_state_blocking: forest_filecoin::interpreter::fvm4: database lookup error: Can't create a valid state tree from the given root. This error may indicate unsupported version. state_root_cid=bafy2bzaceb4rwdcrjd45w4jwkxmjyv6ydlgx4s4f7pvrlv2w3uevwsxqziiaa, state_root_version=unknown 2024-04-23T16:11:00.106157Z INFO forest_filecoin::chain_sync::tipset_syncer: Validating tipset: EPOCH = 1552075, N blocks = 3 2024-04-23T16:11:00.106759Z INFO forest_filecoin::chain_sync::tipset_syncer: Validating tipset: EPOCH = 1552076, N blocks = 1 ```
LesnyRumcajs commented 2 months ago

@elmattic Do you have this snapshot around, or did you upload it anywhere? If not, it's too difficult to reproduce and we should just close it until it surfaces again - then we can upload a snapshot.

elmattic commented 2 months ago

@elmattic Do you have this snapshot around, or did you upload it anywhere? If not, it's too difficult to reproduce and we should just close it until it surfaces again - then we can upload a snapshot.

I don't have the snapshot anymore. Let's close this and reopen it if it shows up.