spacemeshos / post

Spacemesh POST protocol implementation
MIT License
19 stars 20 forks source link

Allow initializing only subset of labels #179

Closed poszu closed 1 year ago

poszu commented 1 year ago

Motivation

It is now possible to initialize a subset of POS data with postcli. The initialization can be split on file boundary (i.e. files (0-N), (N+1 - M), etc.)

Changes

Added 2 new arguments to postcli:

-fromFile int
        index of the first file to init (inclusive)
-toFile int
        index of the last file to init (inclusive). Will init to the end of declared space if not provided.

Added the value of found nonce to metadata in order to be able to select the best nonce. The operator combining separately initialized data will need to manually pick the best CRF nonce if they wish to have the best possible nonce. Details in the updated readme.

poszu commented 1 year ago

An example of overlapping initializations and md5sums of the created data:

~/post/data4

Skip -from - start from the beginning.

go run ./cmd/postcli  -maxFileSize 1048576 -labelsPerUnit 262144 -to 262144 -provider 4294967295 -commitmentAtxId 00037ec8ec25e6d2c00000000000000000000000000000000000000000000000 -id 518e2824769a7e2ca844bdd6045109c875f62c7505944d79139a2c85aaa3de3c -datadir ~/post/data4/
❯ md5sum ~/post/data4/*
c812b7e913df797245fe6dc5b1cdede9  /home/bartosz/post/data4/postdata_0.bin
febee84428bb6bb3a61d44442a4198b1  /home/bartosz/post/data4/postdata_1.bin
66b66cd003a4bc3faf4721b8256e7f9c  /home/bartosz/post/data4/postdata_2.bin
22197e3c25e7491e714860e13a6fb1e4  /home/bartosz/post/data4/postdata_3.bin

~/post/data5

❯ go run ./cmd/postcli  -maxFileSize 1048576 -labelsPerUnit 262144 -from 65536 -to 327680 -provider 4294967295 -commitmentAtxId 00037ec8ec25e6d2c00000000000000000000000000000000000000000000000 -id 518e2824769a7e2ca844bdd6045109c875f62c7505944d79139a2c85aaa3de3c -datadir ~/post/data5/
❯ md5sum ~/post/data5/*
febee84428bb6bb3a61d44442a4198b1  /home/bartosz/post/data5/postdata_1.bin
66b66cd003a4bc3faf4721b8256e7f9c  /home/bartosz/post/data5/postdata_2.bin
22197e3c25e7491e714860e13a6fb1e4  /home/bartosz/post/data5/postdata_3.bin
af906c228c223ddecabf120ce76c7cc3  /home/bartosz/post/data5/postdata_4.bin
44dd75008fd9e6b7c99b50dd55e873ab  /home/bartosz/post/data5/postdata_metadata.jso
poszu commented 1 year ago

I changed the interface. It now accepts -fromFile <idx> and -toFile <idx> - file indices to initialize. This is less error-prone and allows splitting init at a file boundary.

dshulyak commented 1 year ago

does it allow to set file size?

pigmej commented 1 year ago

@dshulyak --max-file-size ? :)