search

input-output-hk / rust-byron-cardano

rust client libraries to deal with the current cardano mainnet (byron / cardano-sl)
MIT License
264 stars 75 forks source link

Upstreaming OBFT stuff #693

Closed vsubhuman closed 5 years ago

vsubhuman commented 5 years ago

Upstreaming all the changes we have done in the fork. Includes:

  1. OBFT processing (does not include OBFT signature validation)
  2. Rollback handling
  3. Special new index for epoch-lengths and loose blocks
  4. Changes in epoch pack files to include epoch length and flags for whether it's an EBB epoch or not
  5. Changes in storage to support block-by-height and status endpoints in the bridge ("status" returns information about currently known local and network tips)
vsubhuman commented 5 years ago

@disassembler

vsubhuman commented 5 years ago

Notes to self:

vsubhuman commented 5 years ago

Discovered that we, after all, need to fix loose blocks deleting at rollbacks, and also need to close some rollback scenarios that we made possible in the code, but which are impossible in Cardano. @rooooooooob is working on it atm. Will activate PR after this.

vincenthz commented 5 years ago

@vantuz-subhuman there's no reason to explicitly delete loose blocks that are not used anymore, unless you really really want to delete them now. the storage design is that eventually this is taken care by packing/gc (think of git)

vsubhuman commented 5 years ago

the storage design is that eventually this is taken care by packing/gc (think of git)

🤔 Interesting. I thought loose blocks only deleted when epoch becomes stable and gets packed, but then we go from tip back in history to link the epoch blocks, so blocks that were forked out would get lost.

Seems I might be missing something there then. Will check it out, thank you! But then we are still waiting for some small fixes in rollback logic, to not allow it to go ways it shouldn't.

vincenthz commented 5 years ago

All the code to do that is not necessary in place (can't remember the details right now), but it's not a problem to have unreferenced blocks on disk, just like a commit in git that is not used anymore, it is eventually gc

vincenthz commented 5 years ago

I started looking at the PR, but this likely suffer the same too-many-things-at-once problem of the first one.

Please cut the obvious improvements, refactoring and adjustment into multiples PRs, leaving the more sizeable parts as logical improvement, and this would help massively get this push through