yacovm
commented
1 year ago If a block is replicated to only a minority and a new leader is selected then the WAL can be overwritten. Also, the WAL is periodically pruned, as mentioned by Kostas in the video.
If we only use the ledger, then we need to be able to prune it and overwrite it on demand from the Raft library. If we only use the WAL, then we need it to be indexable for search (to present a ledger API).
Current implementation of Ordering Service Node(OSN) with raft consensus type manages the blocks data in 2 different places
As the same block contents written into 2 separate file system locations: 1) Raft library writes into WAL while consenting on the block 2) OSN writes into ledger post the consensus & commit into its ledger, It requires more than one filesystem IO and additional filesystem storage.
If we make the OSN and consensus library to use/refer the same block data, it would reduce the filesystem IO and additional storage required which may provide performance advantage.
The same view point was called out previously during the design call: https://drive.google.com/file/d/11qDdi0-93f7CwYxsdmTl9BWITYy5VUtl/view . It is at the 38th minute of the replay(38:20 —— 40:38 time window)
It is interesting to understand the alternate implementation and its performance benefits. Opening this issue to get the community view points, evaluate and propose the alternate approach.