timescale / timescaledb

An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
https://www.timescale.com/
Other
16.83k stars 852 forks source link

[feature-request]: reorder policy that wont blocks writes #6802

Open lb-ronyeh opened 3 months ago

lb-ronyeh commented 3 months ago

What type of issue is this?

Locking issue

What subsystems and features are affected?

Background worker, Policy, Query executor

What happened?

according to the docs, reorder policy wont lock queries (reads, what abouts writes ?)

https://docs.timescale.com/api/latest/hypertable/reorder_chunk/

TimescaleDB version affected

2.12.0

PostgreSQL version used

14.6

What operating system did you use?

Ubuntu 21.04 x64,

What installation method did you use?

Deb/Apt

What platform did you run on?

On prem/Self-hosted

Relevant log output and stack trace

SELECT * FROM _timescaledb_config.bgw_job
 return :

{
"SELECT * FROM _timescaledb_config.bgw_job\n": [
    {
        "id" : 2,
        "application_name" : "Error Log Retention Policy [2]",
        "schedule_interval" : "1 mon",
        "max_runtime" : "01:00:00",
        "max_retries" : -1,
        "retry_period" : "01:00:00",
        "proc_schema" : "_timescaledb_functions",
        "proc_name" : "policy_job_error_retention",
        "owner" : "postgres",
        "scheduled" : true,
        "fixed_schedule" : true,
        "initial_start" : "2000-01-01 00:00:00+00",
        "hypertable_id" : null,
        "config" : "{\"drop_after\": \"1 month\"}",
        "check_schema" : "_timescaledb_functions",
        "check_name" : "policy_job_error_retention_check",
        "timezone" : null
    },
    {
        "id" : 1,
        "application_name" : "Telemetry Reporter [1]",
        "schedule_interval" : "24:00:00",
        "max_runtime" : "00:01:40",
        "max_retries" : -1,
        "retry_period" : "01:00:00",
        "proc_schema" : "_timescaledb_functions",
        "proc_name" : "policy_telemetry",
        "owner" : "postgres",
        "scheduled" : true,
        "fixed_schedule" : false,
        "initial_start" : null,
        "hypertable_id" : null,
        "config" : null,
        "check_schema" : null,
        "check_name" : null,
        "timezone" : null
    },
    {
        "id" : 1000,
        "application_name" : "Reorder Policy [1000]",
        "schedule_interval" : "336:00:00",
        "max_runtime" : "00:00:00",
        "max_retries" : -1,
        "retry_period" : "00:05:00",
        "proc_schema" : "_timescaledb_functions",
        "proc_name" : "policy_reorder",
        "owner" : "postgres",
        "scheduled" : true,
        "fixed_schedule" : false,
        "initial_start" : null,
        "hypertable_id" : 1,
        "config" : "{\"index_name\": \"git_metrics_organization_id_metric_time_idx\", \"hypertable_id\": 1}",
        "check_schema" : "_timescaledb_functions",
        "check_name" : "policy_reorder_check",
        "timezone" : null
    },
    {
        "id" : 1001,
        "application_name" : "Reorder Policy [1001]",
        "schedule_interval" : "336:00:00",
        "max_runtime" : "00:00:00",
        "max_retries" : -1,
        "retry_period" : "00:05:00",
        "proc_schema" : "_timescaledb_functions",
        "proc_name" : "policy_reorder",
        "owner" : "postgres",
        "scheduled" : true,
        "fixed_schedule" : false,
        "initial_start" : null,
        "hypertable_id" : 2,
        "config" : "{\"index_name\": \"pm_metrics_organization_id_metric_time_idx\", \"hypertable_id\": 2}",
        "check_schema" : "_timescaledb_functions",
        "check_name" : "policy_reorder_check",
        "timezone" : null
    }
]}

### How can we reproduce the bug?

```bash
execute writes while reorder policy enabled and running 

evidence of lock

image
nikkhils commented 3 months ago

@lb-ronyeh as mentioned in the documentation, not supporting writes is a limitation of this API. However when compared to the PG CLUSTER command we are already doing a little bit better by allowing READs whereas the PG command blocks both reads and writes.