[dd] enable change ddse from innodb to rocksdb

[luqun]

Summary:

The change is to "upgrade" DD tables from innodb SE to rocksdb SE.

Steps for a DD table involving DDSE change: apply to all DD tables

- create target DD tables into <upgrade> schema by execute "Create table"
- 'logical copy': execute "INSERT INTO <upgrade>.<Table> SELECT * FROM mysql.<Table>" 
- "rename" mysql.<Table> to <temporary>.<Table> and "rename" <upgrade>.<Table> to mysql.<Table>

Workflow for innodb SE to rocksdb SE changes

- create a schema for target DD tables(such as <upgrade>) and create another schema for temporary usage(such as <temporary>
- upgrade dd_properties table
- upgrade other DD tables
- update DD_ENGINE/VERSION property in dd_properties table and  commit changes with one trx
- drop <temporary> schema

The above workflow for INNODB->ROCKSDB DDSE change is finished with two writebatch commit first and one transaction commit

• Two writebatch: see update_myrocks_table_names()
  • The first write batch is to fix/update/rename myrocks internal DD for dd_property table(rename .dd_properties to mysql.dd_properties); after commit, table_exists_in_engine for ROCKSDB will return true; while before commit, table_exists_in_engine for ROCKSDB will return false
  • the second batch is also to fix/update/rename myrocks internal DD for other DD tables.

• One transaction: it hold create tables, "rename" table, "logical copy" and committed in update_versions()

  Whether to do DDSE change is controlled by 3 DDSE values:

• default_dd_storage_engine: It is a mysql variable and its value is defined in CNF file or default value if not specified. Use V1 for reference
• DD_ENGINE property in mysql.properties table: its value is stored during last step of previous DDSE change or during boostrap, Use V2 for reference
• table_exists_in_engine handle API: runtime only value. before open first table(dd_properties), guess that if one SE hold dd_properties table, then this SE might be DDSE. Use V3 for reference

If V1, V2, V3 are equal, then SKIP DD table change; otherwise, try DD table change

During DDSE change, if crash occurs, what's next?

Crash Scenarios during INNODB->ROCKSDB DDSE change

• Crash before first writebatch commit: after crash, during restart, its V1 is ROCKSDB, V2 is INNODB, V3 is INNODB, upgrade DD tables again.
• Crash before second writebatch commit or before Transaction commit: after crash, during restart, its V1 value is ROCKSDB, V2 value is INNODB, V3 value is ROCKSDB, upgrade DD tables again..
• crash after Transaction commit: after crash, during restart, its V1 value is ROCKSDB, V2 value is ROCKSDB and V3 is ROCKSDB, SKIP upgrade DD tables

Remaining issues:

• during ddse change, there a crash. after restart, CNF changed.

[laurynas-biveinis]

@luqun , is this ready for review?

[luqun]

@luqun , is this ready for review? Yes. maybe I shouldn't force push?

[laurynas-biveinis]

@luqun , is this ready for review? Yes. maybe I shouldn't force push?

Force pushes are fine with me

[facebook-github-bot]

@luqun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[luqun]

committed