Reduce pruned node disk space

[sophoah]

Description

Reduce the pruned node disk space

Context

External validator has been struggling with the constant shard 0 disk increase and cost associated to it. As of 21 October, the s0 disk space is about 250GB. For a validating node only, there are no need to keep the full database. @rlan35 confirmed there are no security risk as long as enough blocks are kept so staking information are available

This bounty is to add into the harmony core protocol a new feature that will keep only the last xxxx blocks of data.

this feature will not be used for node that are used as RPC service.

Acceptance Criteria

• [ ] submit PR to https://github.com/harmony-one/harmony/
  • [ ] to update the harmony config file and cli argument to accept a new flag that will enable the feature
  • [ ] to implement the feature
  • [ ] proper unit test developed accordingly
• [ ] working demo of a testnet node showing
  • [ ] active validator BINGO-ing with the old blocks removed from the db
  • [ ] old blocks are being deleted as new one are being added
• [ ] PR approved and deployed to testnet

Reward

[5k USD of tokens in ONE]

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 20809.2489 ONE (5014.65 USD @ $0.24/ONE) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
  • Want to chip in? Add your own contribution here.
  • Questions? Checkout Gitcoin Help or the Gitcoin's Discord
  • $39,552,708.47 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 265 years from now. Please review their action plans below:

1) rhythm-one has started work.

I plan on working on this over the coming week. What requirements are there for unit testing? If a validator is running BLS keys on shard 0 would we ignore this option, or is it safe to remove blocks in that case as well?

Learn more on the Gitcoin Issue Details page.

[sophoah]

@rhythm-one all codes that you are creating should be tested. for the shard 0 question, I believe it should be safe to also delete it. However I will let @rlan35 to confirm that.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 20809.2489 ONE (6543.78 USD @ $0.34/ONE) has been submitted by:

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin's Discord
• $6,933,954.38 more funded OSS Work available on the Gitcoin Issue Explorer

[LuttyYang]

@sophoah I rechecked with @rlan35 again to confirm this bounty. I made the following restrictions.

1. Only validator node with shard 1-3 can use this feature, I add this limit in the hard code.
2. @rlan35 to confirmed that it is safe to keep 8 blocks of epoch data, And we think this does not need to be configured, so I set a constant to set how many blocks of data to keep, currently it's 129600

At present I have completed the local test and I am testing in the testnet

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 20809.2489 ONE (7044.06 USD @ $0.32/ONE) has been submitted by:

1. @luttyyang

@leohchen please take a look at the submitted work:

• PR by @luttyyang

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin's Discord
• $7,258,278.66 more funded OSS Work available on the Gitcoin Issue Explorer