There is a 4000 partition limit for BQ tables, our unified views are already above 3950 days. It is not clear if the table partition limit applies to views.
Note also that our oldest web100 data is from 2009-02-18, which is more than 4100 days in the past:
SELECT min(partition_date), date_diff(current_date(), min(partition_date), day) FROMmeasurement-lab.ndt.web100
(I have not yet explored the 50+ day discrepancy)
We need a long term plan going forward:
Ask to raise the limit
partition by weeks
partition on other units. e.g. server-months or server-years
Use named tables for years, conventional partitioning for days within years
There is a 4000 partition limit for BQ tables, our unified views are already above 3950 days. It is not clear if the table partition limit applies to views.
Note also that our oldest web100 data is from 2009-02-18, which is more than 4100 days in the past:
(I have not yet explored the 50+ day discrepancy)
SELECT min(partition_date), date_diff(current_date(), min(partition_date), day) FROM
measurement-lab.ndt.web100We need a long term plan going forward: