Open AnnaShaleva opened 2 weeks ago
By the way, we finally found duplicates at the end. example:
Chain block height: 6689772
Uploading index files...
Searching for blocks within the range 6680000 6690000, found 14742
Duplicate block 6686840 with oid saved 381yPLFRr9PN1wWKDiLmaR7QvjhxvvYNXFNo7KeGDPfb and new one 5sPVCKbNZwS27w1a61qDqPyXxNfEf4PKpgCvdmuwbisa
and objects differ with checksums - which means its different payloads (blocks):
(base) ekaterinapavlova@MacBook-Air-4 neofs-node % ./bin/neofs-cli object get --file ./res2.bin -r apollo7.morphbits.io:8080 -w /Users/ekaterinapavlova/Workplace/neo-go/panelwallet1.json --cid ANMHF1LcEhf4LsFQRCafst5PdHN1YTvhaZYk1EhLKMPH -t 60s --oid 92vfVNYQgF3gZyJyDKJDGhrkVHas2DYR6jdqSWMvumF7
697 / 697 [=============================================================================================================================================================================] 100.00% 0s
[./res2.bin] Object successfully saved
ID: 92vfVNYQgF3gZyJyDKJDGhrkVHas2DYR6jdqSWMvumF7
CID: ANMHF1LcEhf4LsFQRCafst5PdHN1YTvhaZYk1EhLKMPH
Owner: NSqo6fy6uGRTLQUofh8VkB1r6G7VbMnFmM
CreatedAt: 1
Size: 697
HomoHash: 5bdefd06efbbf0c66003c05dcfaa340441bc09bb06c8d6f1f2ae1291b3fc42ff0dbad5dfa481ebeaa5fcd5f39732379d7cf962a9c33de229abb29ce4462344ae
Checksum: 7ba30f2ba341a7e55cf9d3a1f1334d19e3f75ec1199b39a4225d447c83a27487
Type: REGULAR
Attributes:
Block=6686383
Primary=4
Hash=97d6402f58445fc6cff40c80ea46c52af26e2971d2ba26637b87a551c2565d8f
PrevHash=0aed6b9903e34194759eee7953f2c471ca20aae46f1e999e9e4431bac2b11316
Timestamp=1731570803990 (56841-04-17 20:59:50 +0300 MSK)
ID signature:
public key: 037902671970564dae4366aab65fe1da930c41ce12b1908f99f3e7fa3843f3996d
signature: 9a11fdd7997e0d0b2bfe21f93ca815b683d442c64b90da8c70ced5ac320dfe953fb1f490a52bd9d217628892d9d490738c6832e0306eaae9b28f2cdaf6e2b5a7
(base) ekaterinapavlova@MacBook-Air-4 neofs-node % ./bin/neofs-cli object get --file ./res1.bin -r apollo7.morphbits.io:8080 -w /Users/ekaterinapavlova/Workplace/neo-go/panelwallet1.json --cid ANMHF1LcEhf4LsFQRCafst5PdHN1YTvhaZYk1EhLKMPH -t 60s --oid H5U4i6PxYAwAZHVuEzFKJzs2gJMno6e8zxCqxDiJB2ia
697 / 697 [=============================================================================================================================================================================] 100.00% 0s
[./res1.bin] Object successfully saved
ID: H5U4i6PxYAwAZHVuEzFKJzs2gJMno6e8zxCqxDiJB2ia
CID: ANMHF1LcEhf4LsFQRCafst5PdHN1YTvhaZYk1EhLKMPH
Owner: NSqo6fy6uGRTLQUofh8VkB1r6G7VbMnFmM
CreatedAt: 1
Size: 697
HomoHash: 48d156668021420a6e1dbe2a30986b701cccb098333b95976887623b7d6706173016f309d3c5b8dc6c8efe6fabda0fde66f6a62872d93a25b93b114fa45d0963
Checksum: 717166bec84eba2f53fde4bf6b6a400527b95996c79aee6f93adb7baf8a677ea
Type: REGULAR
Attributes:
Block=6686383
Primary=4
Hash=97d6402f58445fc6cff40c80ea46c52af26e2971d2ba26637b87a551c2565d8f
PrevHash=0aed6b9903e34194759eee7953f2c471ca20aae46f1e999e9e4431bac2b11316
Timestamp=1731570803990 (56841-04-17 20:59:50 +0300 MSK)
ID signature:
public key: 037902671970564dae4366aab65fe1da930c41ce12b1908f99f3e7fa3843f3996d
signature: c989a391f3d0bb33e7b3ad93e5d41afce92957927190e0d1b001bfc57de13ea6b6792a632027e74b004b06bf9b4ec922a430e323ae85986004cb0f69d7a2c4b4
Is your feature request related to a problem? Please describe.
Currently
upload-bin
finds latest incomplete batch of blocks and starts reuploading from the first block in this batch. Due to https://github.com/nspcc-dev/neo-go/issues/3647#issuecomment-2444172577 objects with the same block index but different OIDs may be created using this way of uploading.Describe the solution you'd like
Don't upload the same block twice. Instead, search for the latest incomplete batch, then search for the missing block indexes in this batch and upload only missing blocks.
Describe alternatives you've considered
Leave it as is. It works, but a lot of useless duplicating objects are created which makes SEARCH request slower.