search

spacemeshos / post

Spacemesh POST protocol implementation
MIT License
19 stars 20 forks source link

Disable looking for VRF nonce past initialized data when using `-from/toFile` flag #187

Closed poszu closed 1 year ago

poszu commented 1 year ago

The postcli can be used to initialize only a subset of data using -fromFile and -toFile flags (for example when re-initializing corrupted/missing data on a different machine or splitting initialization between machines).

The current behavior is to continue looking for VRF nonce (which is very likely to happen if initializing only a subset). It should be possible to disable looking for VRF nonce past the requested size (either automatically when -fromFile or -toFileflag is used or via a flag like -disableEagerVrfSearch or something like that.

fasmat commented 1 year ago

The reason we are looking past the range of initialization is because the range might not include a valid VRF nonce. The probability for this is small - only ~ 0.1% probability of not finding a valid nonce within the range during initalization.

Those 0.1% of smeshers will then have no VRF nonce in any of their postmeta_data.json files from the individual chunks they initialized.

poszu commented 1 year ago

@fasmat yes, but the problem here is different. The behavior is not right when a user wants to initialize only a subset, say 1 file out of 1000 -then the risk of not finding the nonce is very high. In such a scenario, the user might not want to continue looking for the nonce past the said range of labels.

fasmat commented 1 year ago

I'm not denying that, I'm just trying to say that we then need to handle this special case where a user initialized in chunks but none of the chunks had a valid VRF.

poszu commented 1 year ago

This special case needs to be handled manually by the operator. The postcli has no way of knowing about the initialization process of the other files (they will be initialized on a different machine most likely). The how-to should land in the readme. Something along the lines:

If you initialized all the files and still don't have a VRF nonce, initialize the last file again without -disableEagerVrfSearch (in a directory with this file to avoid re-init

// in directory with postdata_lastFileIndex.bin
postcli -fromFile <lastFileIndex> <all other args>...

it will continue to look for VRF nonce automatically.