timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
17.49k stars 879 forks source link

Support space partitions per distinct value #544

Open mbande opened 6 years ago

mbande commented 6 years ago

I have a table with an integer field that takes limited values 1,2,3. and I want to space partition by this field, one partition for each value, i.e. to create 3 partitions and use value as exact partition id. searching the documentation, I cant figure out an easy solution for this.

LeeHampton commented 6 years ago

You can specify a custom partitioning function (see explanation for the partitioning_func argument here). With that you can control which partition a given element gets sent to. You can't directly control the exact partitioning id, however.

mbande commented 6 years ago

So, to enforce 1 partition per field value, i should initialize hypertable with 3 space partition and use an identity (f(x)->x) partitioning function, right?

erimatnor commented 6 years ago

@mbande the function can't be the identity function. The purpose of the partitioning function is to map a value onto a int32 (positive) key space. That key space is then divided evenly across the number of partitions. So, for three partitions and an input value of 1, the partitioning function needs to output a positive int32 between 0 and INT32_MAX/3. For 2 and 3 it would be analogous but falling instead into the two remaining ranges.

bartpeeters commented 5 years ago

We have the exact same usecase, any plans to add this as a new feature? More generic would be a possibility to dynamically add new partitions, just how a hypertable makes new partitions based on time ranges, but in this case based on unique ids.

adrianog commented 5 years ago

Where can I find more info about partitioning_func? An example that goes through defining the function and passing it to create_hypertable?

I only see this entry in the documentation: SELECT create_hypertable('conditions', 'time', 'location', 4, partitioning_func => 'location_hash');

erimatnor commented 4 years ago

Related to #563

muntdan commented 10 months ago

Any update on this please?

muntdan commented 10 months ago

I belive this might work: If you want let's say 100 Tenant/partition and use smallint to count them from 0 to 99. Because 2147483647/100 = 21474836, the function would look like this:

CREATE OR REPLACE FUNCTION SinglePartition(tenant anyelement) RETURNS 
integer AS 
$$
BEGIN
 RETURN tenant::smallint * 21474836 + 1;
END;
$$ 
LANGUAGE plpgsql IMMUTABLE;