timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
16.82k stars 852 forks source link

Fix catalog corruption on Continuous Aggregate #7023

Closed fabriziomello closed 2 weeks ago

fabriziomello commented 3 weeks ago

Unfortunately the code-path for ALTER TABLE...{RENAME | SET SCHEMA} over a Continuous Aggregate was not handled very well in the process utility hook leading to a catalog corruption because it was not updating properly the internal metadata information.

Fixed it by properly update the catalog information.

Disable-check: force-changelog-file

codecov[bot] commented 3 weeks ago

Codecov Report

Attention: Patch coverage is 80.76923% with 5 lines in your changes missing coverage. Please review.

Project coverage is 81.68%. Comparing base (59f50f2) to head (21da1b7). Report is 210 commits behind head on main.

Files Patch % Lines
src/process_utility.c 80.76% 2 Missing and 3 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #7023 +/- ## ========================================== + Coverage 80.06% 81.68% +1.61% ========================================== Files 190 199 +9 Lines 37181 37009 -172 Branches 9450 9674 +224 ========================================== + Hits 29770 30230 +460 + Misses 2997 2877 -120 + Partials 4414 3902 -512 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

RobAtticus commented 3 weeks ago

There are more cases that need to be checked, e.g., SET SCHEMA also causes inconsistent catalog:

tsdb=> ALTER TABLE foo_one SET SCHEMA rollups;
ALTER TABLE
tsdb=> SELECT * FROM _timescaledb_catalog.continuous_agg WHERE user_view_name = 'foo_one';
 mat_hypertable_id | raw_hypertable_id | parent_mat_hypertable_id | user_view_schema | user_view_name |  partial_view_schema  | partial_view_name | bucket_width |  direct_view_schema   | direct_view_name | materialized_only | finalized
-------------------+-------------------+--------------------------+------------------+----------------+-----------------------+-------------------+--------------+-----------------------+------------------+-------------------+-----------
               237 |                29 |                          | public           | foo_one        | _timescaledb_internal | _partial_view_237 |  86400000000 | _timescaledb_internal | _direct_view_237 | t                 | t
(1 row)
fabriziomello commented 3 weeks ago

There are more cases that need to be checked, e.g., SET SCHEMA also causes inconsistent catalog:

tsdb=> ALTER TABLE foo_one SET SCHEMA rollups;
ALTER TABLE
tsdb=> SELECT * FROM _timescaledb_catalog.continuous_agg WHERE user_view_name = 'foo_one';
 mat_hypertable_id | raw_hypertable_id | parent_mat_hypertable_id | user_view_schema | user_view_name |  partial_view_schema  | partial_view_name | bucket_width |  direct_view_schema   | direct_view_name | materialized_only | finalized
-------------------+-------------------+--------------------------+------------------+----------------+-----------------------+-------------------+--------------+-----------------------+------------------+-------------------+-----------
               237 |                29 |                          | public           | foo_one        | _timescaledb_internal | _partial_view_237 |  86400000000 | _timescaledb_internal | _direct_view_237 | t                 | t
(1 row)

Yep... I saw it after send the PR... fixing it as well

fabriziomello commented 3 weeks ago

@mkindahl @antekresic after the today discussion I've changed the approach to make ALTER TABLE ... {SET SCHEMA|RENAME} works on CAggs instead of block it... if you're fine with the second commit I'll provide the necessary changes on the PR description and commit message.