MystenLabs / sui

Sui, a next-generation smart contract platform with high throughput, low latency, and an asset-oriented programming model powered by the Move programming language
https://sui.io
Apache License 2.0
6.26k stars 11.21k forks source link

[sui-node] Node synced via RocksDB snapshot node missing many dynamic fields of objects #19824

Open shrimpwtf opened 1 month ago

shrimpwtf commented 1 month ago

Steps to Reproduce Issue

made a new node and synced it fully via a rocksdb snapshot. its synced to the same state as the public rpc im using for comparison. when I grab object 0xf699e7f2276f5c9a75944b37a0c5b5d9ddfd2471bf6242483b03ab2887d198d0, both say the size is 4066 however when grabbing the dynamic fields of it the public rpc grabs all the objects meanwhile mine only grabs 478 of them.

I've got these settings enabled as below, from the docs explanations this only wipes historical versions of objects and shouldn't affect this.

`enable-event-processing: true enable-index-processing: true

authority-store-pruning-config: num-latest-epoch-dbs-to-retain: 3 epoch-db-pruning-period-secs: 3600 num-epochs-to-retain: 1 max-checkpoints-in-batch: 10 max-transactions-in-batch: 1000 pruning-run-delay-seconds: 60`

Expected Result

all dynamic fields are returned

Actual Result

only 10% of the objects were returned.

System Information

stefan-mysten commented 1 month ago

Can you explain what do you mean by size: 0xf699e7f2276f5c9a75944b37a0c5b5d9ddfd2471bf6242483b03ab2887d198d0, both say the size is 4066? You mean the number of dynamic fields that this object has?

shrimpwtf commented 1 month ago

Can you explain what do you mean by size: 0xf699e7f2276f5c9a75944b37a0c5b5d9ddfd2471bf6242483b03ab2887d198d0, both say the size is 4066?

You mean the number of dynamic fields that this object has?

Correct. At the time of testing was roughly around this. I'm trying a snapshot sync with full indexes to see if this helps

shrimpwtf commented 1 month ago

Can you explain what do you mean by size: 0xf699e7f2276f5c9a75944b37a0c5b5d9ddfd2471bf6242483b03ab2887d198d0, both say the size is 4066? You mean the number of dynamic fields that this object has?

So I've just synced the rocksdb snapshot with the indexes (1.3tb of them too), it's a bit better, grabbing 3699 of them however there's currently 4203 in this object so it's still missing a bunch. The node is fully upto sync. This is definitely a big problem. CleanShot 2024-10-12 at 12 41 18@2x

shrimpwtf commented 1 month ago

Public and my rpc seem to give same result for the above now, however other objects still have discrepancies. CleanShot 2024-10-12 at 13 05 51@2x