timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
16.8k stars 852 forks source link

Add index scan to INSERT DML decompression #7048

Closed antekresic closed 1 week ago

antekresic commented 1 week ago

In order to verify constraints, we have to decompress batches that could contain duplicates of the tuples we are inserting. To find such batches, we use heap scans which can be very expensive if the compressed chunk contains a lot of tuples. Doing an index scan makes much more sense in this scenario and will give great performance benefits.

Additionally, we don't want to create the decompressor until we determine we actually want to decompress a batch so we try to lazily initialize it once a batch is found.

codecov[bot] commented 1 week ago

Codecov Report

Attention: Patch coverage is 82.89474% with 39 lines in your changes missing coverage. Please review.

Project coverage is 81.86%. Comparing base (59f50f2) to head (0685126). Report is 222 commits behind head on main.

Files Patch % Lines
tsl/src/compression/compression.c 83.03% 16 Missing and 22 partials :warning:
src/nodes/chunk_dispatch/chunk_insert_state.c 75.00% 0 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #7048 +/- ## ========================================== + Coverage 80.06% 81.86% +1.79% ========================================== Files 190 200 +10 Lines 37181 37297 +116 Branches 9450 9724 +274 ========================================== + Hits 29770 30533 +763 + Misses 2997 2861 -136 + Partials 4414 3903 -511 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.