maxtepkeev / architect

A set of tools which enhances ORMs written in Python with more features
Other
391 stars 57 forks source link

Get the partition table name for a specific partition value. #37

Open dvfabbri opened 7 years ago

dvfabbri commented 7 years ago

I want to bulk insert rows into a specific partition which all have the same date. Assume that data are partitioned by month on date_column. Does the architect API allow for:

p = get_partition(column_name='date_column', value='2015-01-01')

then run: "copy from '/my/file.txt' to %s" % p.table_name

Perhaps I am unclear on how to use the get_partition API. Can you provide an example?

maxtepkeev commented 7 years ago

Unfortunately there is no API to get the partition name right now, neither in PostgreSQL nor in MySQL. In MySQL it's fairly easy to implement using existing code, but for PostgreSQL the partition name is constructed dynamically inside the trigger so we have to think about how to implement it properly in Architect, because your use case is perfectly valid and we need to add this API in one of the future versions.

The only thing that I can recommend you to do right now is to hardcode the partition name in your code if it's the one time operation, or if you plan to do this on a regular basis you can have a look at the generated partition names to get idea of the naming scheme and to implement a small function in Python somewhere in your code which will accept a date and return a generated partition name from it.

dvfabbri commented 7 years ago

Thanks. Can I create a new partition through the API or do I need to insert one row into the master table?

maxtepkeev commented 7 years ago

In MySQL you should have both options, in Postgres you need to insert one row into the table for partition to be created, because everything is managed through the triggers.