Immudb stuck if PKILL signal is received

[mmeloni]

Could appen if immudb receives a PKILL or a bad crash that merkle tree elements are not flushed in badger. So this cause that the badger index will be greater than the merkle tree one. The reason is that treeLayerWidth https://github.com/codenotary/immudb/blob/master/pkg/store/treestore.go#L57 is used to set the tree width(t.w) , by counting every treeNodes by layer. But treenodes that corresponds at treeStoreValue at t-x are not presents.

Entering in this cause an infinite loop

func (t *treeStore) WaitUntil(index uint64) {
    for {
        t.RLock()
        if t.w >= index+1 {
            t.RUnlock()
            return
        }
        t.RUnlock()
        time.Sleep(time.Microsecond)
    }
}

In order to avoid this all badger elements that are not linked properly with a merkle tree entries should be truncated at the db startup. Or should we rebuilt them? Requirements: https://github.com/codenotary/immudb/issu themes/39

[mmeloni]

Fix with replayer