tikv / raft-engine

A persistent storage engine for Multi-Raft log
Apache License 2.0
565 stars 88 forks source link

overwrite panic during recovery #250

Closed tabokie closed 2 years ago

tabokie commented 2 years ago

Panic message: attempt to overwrite compacted entries in xx.

This panic should be elided during parallel recovery. Consider this case:

file A: [index=1][index=2] file B: [index=3][index=2][index=3]

It would raise a false panic if we recovery file B alone and try to merge it with file A afterwards.

Workaround:

  1. Disable parallel recovery by setting recovery_threads to 1
  2. Repair the log files with this script:
fn filter_append(id, first, count, rewrite_count, queue, ifirst, ilast) {
  if id == rid && first + count - 1 >= ifirst {
    return 2;  // discard existing
  }
  0
}
fn filter_compact(id, first, count, rewrite_count, queue, compact_to) {
  0
}
fn filter_clean(id, first, count, rewrite_count, queue) {
  0
}
mayjiang0203 commented 1 year ago

/label affects-5.4 /label affects-6.0 /label affects-6.1