search

timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
17.84k stars 884 forks source link

[Enhancement]: Delete by compression segment id #6068

Closed nluangroup closed 1 month ago

nluangroup commented 1 year ago

What type of enhancement is this?

Performance

What subsystems and features will be improved?

Compression

What does the enhancement do?

When setting up compression you can choose a column to segment by to improve performance of where clauses. But why can't we delete by segment id?

Use case: we have segmented our data on an dataTypeId. Each type can have an individual retention policy, thus we would like to dump data in compressed chunks based on the dataTypeId. Our current solution is to decompress the chunk, then delete the expired data, end then recompress it.

Implementation challenges

No response

aarondglover commented 1 year ago

@nluangroup

What version of TSDB are you using? Compressed chunks now support update/delete implicitly. So should negate the requirement to decompress, delete and recompress.

Am I missing something? Is the performance of that deletion poor? Query planner not using index for the delete?

svenklemm commented 1 month ago

This has been added with 2.17.0