search

timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
17.78k stars 886 forks source link

[Feature]: Global access to timescaledb.max_tuples_decompressed_per_dml_transaction #7118

Closed ksuden closed 3 months ago

ksuden commented 3 months ago

What problem does the new feature solve?

I cannot upsert data into compressed chunks or edit records in compressed chunks with packages such as pangres due to not having global access to the timescaledb.max_tuples_decompressed_per_dml_transaction setting. Users are prompted to set this setting to 0 to bypass the setting in the session, but this does not work if you are using a package such as pangres, since pangres constructs the query. TimescaleDB does not allow you to increase the limit or set it to 0 globally in the sql or in the managed service UI. This leaves no other option but to decompress the data first manually, upsert, and then recompress, which defeats the purpose of the semi-recent feature allowing inserts and updates in chunks. The user should have control over this variable and be allowed to consider their resources and make this decision themselves.

What does the feature do?

The feature allows users to set the max_tuples_decompressed_per_dml_transaction setting themselves globally

Implementation challenges

Give managed service users the ability to set max_tuples_decompressed_per_dml_transaction globally via the UI

alexeyklyukin commented 3 months ago

Hi @ksuden, thank you for the report, we'll add this parameter to the UI

You can already change it via ALTER DATABASE ... SET timescaledb.max_tuples_decompressed_per_dml_transaction TO 0; (or any other value) if you like.

To get such requests answered faster, please, refer to cloud support in the future; this repository deals first and foremost with the Postgres database extension.