search

gluent / goe

GOE: a simple and flexible way to copy data from an Oracle Database to Google BigQuery.
Apache License 2.0
8 stars 2 forks source link

Better default partition granularity on BigQuery #196

Open nj1973 opened 3 weeks ago

nj1973 commented 3 weeks ago

On BigQuery we default to 'D' daily partitions. That doesn't really make much sense. We should default to 'M' instead.

We could also think about making this adaptive. If the frontend RDBMS is partitioned on the same column then why not look at the gaps between high values and estimate whether to use 'D', 'M' or 'Y'.

nj1973 commented 2 days ago

I've decided against making the granularity adaptive for this issue, just get that better default in.