search

apavlo / h-store

H-Store Distributed Main Memory OLTP Database System
https://hstore.cs.brown.edu
GNU General Public License v3.0
566 stars 177 forks source link

Live Migration #26

Open apavlo opened 12 years ago

apavlo commented 12 years ago

In this project, students will implement for live migration in H-Store. This will allow a new node to be added to an H-Store cluster without having to take the entire database offline.

  1. Implement a new AbstractHasher that provides consistent hashing. This will allow for tuples to be redirected to the newly inserted node's partitions without having to reorganize the entire database.
  2. Create a new RPC method in the HStoreCoordinator that allows the system to broadcast updates for the system catalog on all nodes to insert the new partition information.
  3. Investigate, design, and implement a live migration scheme. This will essentially move tuples from one or more partitions to a newly added node in the cluster. This must be done in a non-blocking manner to minimize the impact on the database system as it continues to process transactions. The system must figure out whether the new node should get partitions from a single machine or multiple machines.

Students will use H-Store's built-in benchmarks to simulate an overloaded database cluster and then trigger a node to be added to the system. Students should measure how long it takes for their system to complete the migration, as well as compare the performance of the system before, during, and after the migration.

apavlo commented 12 years ago

@yangzou started on this project in the spring semester. The Consistent Hashing portion is partly done in the migration branch. This work is in conjunction with Aaron Elmore at UCSB.