search

celestiaorg / celestia-node

Celestia Data Availability Nodes
Apache License 2.0
932 stars 926 forks source link

DHT Reproviding #2

Closed Wondertan closed 3 years ago

Wondertan commented 3 years ago

https://github.com/lazyledger/lazyledger-core/pull/375 disables providing in Bitswap here. Unfortunately, this options also disables reproviding.

Reproviding is basically re-execution of providing. Another important conceptual part of DHT is that its entries are not persistent and cleaned up through time, thus there should be some logic that automatically renews entries on DHT.

As https://github.com/lazyledger/lazyledger-core/pull/375 disables it automatically we need to enable it manually. Luckily though, we can implement our own strategy for reproviding which only reprovides roots and not all the cids in blockstore, as IPFS excessively do.

Importantly, this is required for the network to work properly, otherwise recently proposed blocks won't be available after ~12 hours. Thus, this needs to be part of celestiaorg/celestia-core#381

liamsi commented 3 years ago

This is related to celestiaorg/celestia-core#394 (and also somewhat to celestiaorg/celestia-core#428) and the story around providing on received data and regular re-providing should be considered together.

liamsi commented 3 years ago

Moving this issue over to the celestia-node as it might still be interesting to have it there for context.

Wondertan commented 3 years ago

DHT Reproviding here is in the context of content or chain blocks. We all agree that DHT and its requirement for reproviding sharded data are not feasible at scale for our use case. Therefore, we can close this.