search

timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
16.83k stars 852 forks source link

Add table mappings to compression settings #6990

Open erimatnor opened 1 month ago

erimatnor commented 1 month ago

Refactor the compression settings metadata table to include a mapping from a chunk's relid to its compressed chunk's relid.

Adding this mapping makes compression settings the main metadata table for compression-related information, while decoupling it from chunk metadata. This simplifies the code that looks up compression metadata as it no longer requires first looking up the corresponding compressed chunk.

The new compression settings is a step towards removing a chunk's compression table from the chunk metadata. In other words, the "compressed chunk" will no longer be a chunk, just a relation associated with the regular chunk via compression settings.

Disable-check: force-changelog-file

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 96.51163% with 3 lines in your changes missing coverage. Please review.

Project coverage is 81.78%. Comparing base (59f50f2) to head (b9df9ea). Report is 196 commits behind head on main.

Files Patch % Lines
tsl/src/compression/compression.c 91.30% 0 Missing and 2 partials :warning:
tsl/src/compression/create.c 83.33% 0 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #6990 +/- ## ========================================== + Coverage 80.06% 81.78% +1.71% ========================================== Files 190 199 +9 Lines 37181 37038 -143 Branches 9450 9675 +225 ========================================== + Hits 29770 30290 +520 + Misses 2997 2861 -136 + Partials 4414 3887 -527 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

erimatnor commented 4 weeks ago

It looks like this PR might need a bit more work. If we're going to store info about compressed and uncompressed assignments in the compression_settings table, we should take it out of the chunk catalog table in this same update. Storing the same thing in two places can lead to confusion and mistakes. We want to keep things tidy and have one source of truth for this information.

While I agree we should get rid of the metadata in the chunk catalog table, I think we should do this in increments. That's a pretty heavy lift and has other consequences on dependent tables, like compression_chunk_size, etc.

This change does not mean we will store the same information in multiple places apart from an extra ID. I think we can live with this in the short term until the other changes are merged too.