MyRocks may fail to restart after crash during ALTER TABLE

[yoshinorim]

How to reproduce:

create table r1 (id int primary key, value text) engine=rocksdb; insert into r1 values (1, 1), (2,2), (3,3), (4,4), (5,5), (6,6); alter table r1 add column value2 text; -> then, crash mysqld after committing into data dictionary (abort after handler::ha_create() to create a working table).

After crash, there should be temporary frm file, and the temporary table info should be committed into MyRocks data dictionary. Then try to restart -- it fails because the frm file is removed but it remains in our dictionary.

2017-07-27 12:09:01 1441804 [Warning] RocksDB: Schema mismatch - Table test.#sql-15c445_8 is registered in RocksDB but does not have a .frm file
2017-07-27 12:09:01 1441804 [ERROR] RocksDB: Problems validating data dictionary against .frm files, exiting
2017-07-27 12:09:01 1441804 [ERROR] RocksDB: Failed to initialize DDL manager.

Since temp tables are wiped on restart, tmp dictionary info should be wiped too.

[mickvav]

Well, I've got into situation where I do have temporary file in data directory, but Myrocks complains that I have no one after crash.

[mickvav]

As for me - just removing of this check solved the issue.

[lth]

A similar problem exists in InnoDB, and they suggest making your own frm file:

https://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting-datadict.html

This problem may be difficult to fix until we move to 8.0.

[mickvav]

Well, as far as I've understood the logic of this piece of code, it does two things:

1. Takes list of tables from myrocks's internals

2. Reads directory for all .frm files, skipping temporary ones (this stanza is what I've disabled in PR)

Than It removes all things found by (2) from list from (1), crashing and leaving you to do whatever you whish to when there still exists anything or when element from first list has no pair in second.

Thus, I suggest that to have more clear recovery path, it would be much better if there would be some option to switch algorithm from "die on any error" behaviour to "clean whatever possible on error and die only if you can't" one.

In this alternate state, which should be explictly marked as "Yo may loose your data if this is enabled" in documentation, it should remove all .frm's not having internal representation, and removing all internal datastructures, describing tables with no .frm (or, if it's possible, re-create .frm file by data from inside).