search

cmu-db / peloton

The Self-Driving Database Management System
http://pelotondb.io
Apache License 2.0
2.03k stars 623 forks source link

Catalog updates for checkpoints #1383

Closed ksaito7 closed 6 years ago

ksaito7 commented 6 years ago

This PR updates some issues related to catalog stuff for checkpoints.

- Related Issues

  1. 1319 Add column length in ColumnCatalogObject and pg_attribute

  2. 1320 Catalog OID collision in checkpoint recovery

  3. 1373 Unexpected catalog object might be obtained from CatalogCache

  4. 1374 Can't recover tile group from LayoutCatalog in checkpoint recovery

- Fix Points

  1. Add column_length feild within pg_attribute and ColumnCatalogObject, and modify related functions in ColumnCatalog.
  2. Add UpdateOid() function within catalogs using oid_ in order to avoid collision between catalog values added by system and users. And add OID_FOR_USER_OFFSET (= 10000) variable within catalog_default.cpp to be used for UpdateOid() function. These are called in last of bootstraps of Catalog and SystemCatalog.
  3. Add database oid/name argument within GetChachedTableObject() function and GetChachedIndexObject() function in order to avoid to acquire an unexpected table from other database. Also fix related functions, GetTableObject() in TableCatalog and GetIndexObject() in IndexCatalog.
  4. Modify Layout stuff to recover layout information for default layout of DataTable and layout of TileGroup.
    1. Modify Layout constructor Layout(num_columns, layout_type) to set appropriate layout oid.
    2. Modify CreateTable() and CreateLayout() in Catalog class to insert all Layout information including row layout and column layout into pg_layout, LayoutCatalog. These avoid to insert same layout. And also modify DropLayout() in Catalog to recreate default layout in pg_catalog in case that it is deleted.
    3. Add default_layout_oid field within pg_table and TableCatalogObject, and modify related functions in TableCatalog. To support for updating default layout, add UpdateDefaultLayoutOid() function within TableCatalog. It is used in CreateDefaultLayout() function and DropLayout() function.
coveralls commented 6 years ago

Coverage Status

Coverage increased (+0.06%) to 77.111% when pulling 05e042c129f9b4e78aa7bbcea550591528a2e817 on ksaito7:catalog_update into e53e5b417900060aa164ef08ad8e52be31f40639 on cmu-db:master.

ksaito7 commented 6 years ago

@aaron-tian Thanks for a lot of comments! I modified them and reply each comments. Please check them again.

db-ol commented 6 years ago

Thanks for the nice work. IsHybridStore() could be added and the type of GetColumnOffset() could be changed if you think it's a good idea, all the others LGTM.

ksaito7 commented 6 years ago

I update them you mentioned. thanks.