citusdata / pg_shard

ATTENTION: pg_shard is superseded by Citus, its more powerful replacement
https://github.com/citusdata/citus
GNU Lesser General Public License v3.0
1.06k stars 63 forks source link

Support composite types / enumeration types / varchar for partition columns #92

Closed onderkalaci closed 9 years ago

onderkalaci commented 9 years ago

Hey @jasonmp85 ,

This PR aims to handle how we find the operatorId for the partition columns. Previously, we find operators by directly using the partition column's type. However, it was wrong. We need to find operators by using the partition column's opfamily input type. The reason is that we use "get_opfamily_member()" to get the operators. As it name implies, that function finds the operators for opfamily. So, providing the opclass' input type makes more sense.

With this PR, I tested with different partition columns (distribute table, test INSERT/SELECT/UPDATE):

Test No Test Type STATUS
1) Test with varchar PASSED
2) Test with enum types PASSED
3) Test with basic types (text) PASSED
4) Test with arrays of basic type PASSED
5) Test with arrays of enum PASSED
6) Test with composite types with hash function defined PASSED
7) Test with arrays of composite types with hash function defined PASSED
8) Test with json type PASSED (cannot distribute table since it has no default op class)
9) Test with jsonb type PASSED

This fix is based on the changes on CitusDB (Composite type bug). However, there is a difference on the implementation compared to that fix. On CitusDB, we changed "MakeOpExpression" function too. However, that is not needed for us since we already use hashed(integers) values on "MakeOpExpression". But, when we support range partitioning, we may need to add that change too. I thought adding it now, but that would seem very unrelated.

Fixes #80

onderkalaci commented 9 years ago

@jasonmp85 I had to spend a lot of time on this issue. I had difficulties in creating new composite types which works fine with pg_shard. For documentation purposes, I note what is required for a composite type to work with pg_shard as distribution key:

  1. Create a composite type
  2. Create a comparator function
  3. Create '=' operator with comparator function
  4. Create an operator family using HASH
  5. Create a hash function
  6. Create a default operator class with USING BTREE (this is required for this line ). Also, set BTEqualStrategyNumber function with the operator created at step 3.
  7. Create a default operator class with USING HASH. Also, set support function to the hash function we already defined.
onderkalaci commented 9 years ago

Hey @jasonmp85 ,

I added a new file for tests. My motivation was to that we have to both create shards and execute queries on that shard. So, putting these tests to create_shards.sql or queries.sql separately didn't fit as good as putting them in a separate file.

What do you think? Is it OK ?

jasonmp85 commented 9 years ago

This looks great. Cleaning up and merging.