cybertec-postgresql / db_migrator

Other
21 stars 8 forks source link

Add an option to not partition a table #13

Closed fljdin closed 1 year ago

fljdin commented 1 year ago

I just realized that there is another problem.

When the data are migrated, you just select from the partitioned table, that is, you get all data from all partitions, right? That would fail to load on PostgreSQL if you omitted one partition.

I am not sure what to do about that. I have two ideas:

  1. Change the way data are migrated so that it is partition by partition (that is the hardest solution).
  2. Only provide the option to migrate partitioning with all partitions as they are or not migrate partitioning at all. That would be good enough from my point of view. Then we should do away with the migrate column for partitions. We can either introduce a migrate_partitioning flag on the table or expect that the user deletes the partitions from the migrator table.

_Originally posted by @laurenz in https://github.com/cybertec-postgresql/db_migrator/pull/11#discussion_r1082793566_

laurenz commented 1 year ago

So you are going with the second solution, right?

laurenz commented 1 year ago

I think this is fixed, right?

fljdin commented 1 year ago

I was thinking about a quick control of missing range during partitions creation. But I don't see a range_agg method in the documentation

laurenz commented 1 year ago

range_agg exists, but only from v14 on, I think.

I think my original complaint was about having a migrate attribute on the subpartition table, that it makes no sense to migrate some partitions, but not others. We don't have that now, so I have no more complaint.

fljdin commented 1 year ago

I am not convinced that a migrate_partitioning column could address this feature. What if a user toggles migrate_partitioning to on and do not insert into (sub)partitions staging tables?

A word in README could be enough or maybe examples in a future how-to documentation:

If we want a control with range_agg (next to v14), I thougth that it could be easy to find a unhandled range in partitions scheme, but converting text[] to int8range or daterange depending on key colomn type is very tricky (maybe impossible?)

laurenz commented 1 year ago

I'll be on vacation for the next three weeks, I'll look at it later.