Schema - Layout decoupling

[poojanilangekar]

This PR decouples the logical schema from the physical layout of the data. Previously, each TileGroup contained a vector of schemas for which violates the notion of the storage layer being a physical store. We also had cases where the LayoutTuner modifies the column_map which ended up modifying the schemas vector via the DataTable. There were multiple other instances where the Schema was used to get the layout of a TileGroup while all we needed was the physical mapping between column_ids and a <tile_id, column_offset> pair. This PR now makes it possible to make schema changes like changing column name, adding & dropping columns without making any calls to the storage layer (i.e., It can now be done entirely in the catalog).

Changes:

• Added Layout and LayoutCatalog classes to store the layout in a physical object and the pg_layout.
• Added a default_layotut_ object to each table (replacing default_partition) and tile_group_layout_ to each TileGroup.
• Persist the HYBRID layouts to the pg_catalog table. The two pre-defined layouts, ROW and COLUMN stores are not persisted.
• Added support in the LLVM engine to read the Layout on a per TileGroup while invoking the TableScanTranslator.
• Modified the LLVM updater and inserter to check that we insert new versions only if the TileGroup has a LayoutType::ROW layout.
• Modified LayoutTuner to tune the Layout rather than the column_map. Additionally, the tuning decisions are now transactional.
• Added the pg_catalog to the SystemsCatalog to ensure that the each layout is stored in a catalog with respect its database.
• Minor cleanup in the catalog (some typos and a couple of magic numbers which caused tests to fail).

Testing:

• Modified the LayoutTuner test to ensure it works as expected after the refactor.
• Minor changes in the existing TileGroupLayoutTest and SeqScanTest which ensures that the old engine works as expected.
• Added tests to the TableScanTranslatorTests to replicate the tests in TileGroupLayoutTest and SeqScanTest. The tests check that the LLVM engine can perform scans on tables different layouts and on tables where each TileGroup has a different Layout.
• Added a test to CatalogTests to ensure that the pg_layout stores and retrieves data as expected and its state is transactional.

Disclaimer:

• Each Tile still contains a Schema which will be used by the old engine, the GetValue and SetValue functions and (currently) the LLVM engines to determine the stride in case of a HYBRID layout. I believe we have to continue supporting the GetValue and SetValue functions even after the old engine is deprecated because we will be needing these interfaces in the Catalog and possibly the Optimizer.
• In order to make the new layout change compatible with the LogicalTile generated by the executor engine, I had to reconstruct the Schemas. This isn't very efficient but the idea is that it is currently only used in tests and will soon be deprecated.
• Unlike the other Catalogs (i.e., ColumnCatalog, StatsCatalog), the LayoutCatalog directly returns a Layout rather than a LayoutCatalogObject since the API are designed to be read only. In essence, the layout is immutable because if you were changing the layout, you need to change the physical representation. So it felt redundant to create a read only object to be returned by the catalog.

@mengranwo Can you please review the catalog changes? @pmenon Can you please review the changes to the LLVM engine? Mainly, RuntimeFunctions, Inserter, Updater, TestingCodegenUtil and TableScanTranslatorTests. @pervazea Can you please review the overall changes? I have modified a few APIs in the old engine. The current code doesn't break the existing tests and it seems to function as expected. But please let me know if you think there is anything I need to change. @jarulraj I know you may not have the bandwidth for this, but if possible could you please review the changes to LayoutTuner and LayoutTunerTest. Nobody else has modified that code and it would be great if you could take a look.

[coveralls]

Coverage increased (+0.1%) to 77.551% when pulling 38a561484d4d0dbcc150f33cf7db5981164d3386 on poojanilangekar:master into d68ab719fdd9499f23ac5887f907ed07ce4d2644 on cmu-db:master.

[mengranwo]

@poojanilangekar Hi, this PR looks good for me except for some minor changes involved and some questions I posted in the comment area. I mainly focus on the logic of catalog & its corresponding test cases and I also take a look at the layout.h/layout.cpp, it stands.

And thank you for fix the typo in DEFAULT_SCHEMA_NAME!