ClickHouse Retention Not Honoring SENTRY_EVENT_RETENTION_DAYS Setting

Self-Hosted Version

24.7.1

CPU Architecture

x86_64

Docker Version

27.1.1

Docker Compose Version

2.29.1

Steps to Reproduce

hi I'm using Sentry v24.7.1 self-hosted, with ClickHouse as the storage backend for events. I’ve configured the environment variable SENTRY_EVENT_RETENTION_DAYS=7 in my .env file, expecting events to be deleted after 7 days. However, when I check the ClickHouse tables, I still see data that is 30 days old, indicating that the retention setting is not being respected. these are 4 most size tables inside of clickhouse

table──────────────────────────────────────────────┬─size───────┐
│ spans_local                                        │ 12.62 GiB  │
│ generic_metric_distributions_aggregated_local      │ 5.62 GiB   │
│ generic_metric_distributions_raw_local             │ 3.57 GiB   │
│ transactions_local                                 │ 1.16 GiB   │
│ query_views_log                                    │ 154.69 MiB │
│ generic_metric_sets_local                          │ 137.31 MiB │

and these are latest records inside spans_local

┌─retention_days─┬─────start_timestamp─┬────start_timestamp_precise─┬─start_ms─┬───────end_timestamp─┬──────end_timestamp_precise─┬─end_ms─┬─duration─┐
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.208719 │      208 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.209528 │    208 │        0 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.201476 │      201 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.202353 │    201 │        0 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.206322 │      206 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.207173 │    206 │        0 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.197592 │      197 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.198417 │    197 │        0 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.213196 │      213 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.214833 │    214 │        1 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.452439 │      452 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.454118 │    453 │        1 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.084820 │       84 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.086433 │     85 │        1 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.495588 │      495 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.501052 │    500 │        5 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.170826 │      170 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.172795 │    171 │        1 │
│             30 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.026508 │       26 │ 2024-11-09 12:42:35 │ 2024-11-09 12:42:35.028064 │     27 │        1 │
└────────────────┴─────────────────────┴────────────────────────────┴──────────┴─────────────────────┴────────────────────────────┴────────┴──────────┘

all of them have 30 days retention why? i tried to install it new multiple times but it didn't work.

if you could give me solution for cleanup of Clickhouse really appreciated it

Thanks in advance for any guidance!

Expected Result

clickhouse keeps data for 7 days

Actual Result

clickhouse keeps data for 30 days

Event ID

No response

getsentry / self-hosted