timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
17.61k stars 883 forks source link

Do not decompress batches for COPY with potentially invalid slot #6931

Closed akuzm closed 4 months ago

akuzm commented 4 months ago

We don't have to decompress anything more when we re-lookup the chunk insert state on COPY buffer flush. Moreover, ChunkInsertState.slots[0] is incorrect slot type for decompress_batches_for_insert(), because it is a chunk slot, not a hypertable slot. This can lead to rare errors when the chunk insert states go out of cache.

Just don't do this unnecessary lookup, and move decompression to a separate function for clarity. Add an assertion and test that detect the slot type mismatch on main.

There was a related fix that improved the situation for empty copy buffers, but not for wrong slot type https://github.com/timescale/timescaledb/pull/6117

Fixes https://github.com/timescale/timescaledb/issues/6540

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 66.66667% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 80.88%. Comparing base (59f50f2) to head (3d46450). Report is 162 commits behind head on main.

Files Patch % Lines
src/nodes/chunk_dispatch/chunk_dispatch.c 75.00% 1 Missing and 2 partials :warning:
tsl/src/compression/compression.c 40.00% 0 Missing and 3 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #6931 +/- ## ========================================== + Coverage 80.06% 80.88% +0.81% ========================================== Files 190 199 +9 Lines 37181 37194 +13 Branches 9450 9700 +250 ========================================== + Hits 29770 30084 +314 - Misses 2997 3234 +237 + Partials 4414 3876 -538 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.