search

getsentry / snuba

Search the seas for your lost treasure.
Other
339 stars 57 forks source link

SENTRY_EVENT_RETENTION_DAYS greater than 90 days does not work as expected. #5502

Open Mandalavandalz opened 9 months ago

Mandalavandalz commented 9 months ago

Self-Hosted Version

24.1.1

CPU Architecture

x86_64

Docker Version

24.0.2

Docker Compose Version

2.0.1

Steps to Reproduce

We are updating Sentry from 9.1.2 to the latest version and want to keep all issues. After following all the hard stops 9.1.2 -> 21.1.0 -> 21.5.0 -> 21.6.3 -> 23.6.2 -> latest and setting SENTRY_EVENT_RETENTION_DAYS=3650 in .env during the update from 9.1.2 to 21.1.0, the Sentry update script correctly processes all issues, which then go into the ClickHouse table sentry_local. When counting the table, it appears that all are in there, but they cannot be selected from the WebUI, only with 90 days back (the default value of SENTRY_EVENT_RETENTION_DAYS). From 21.1.0 -> 21.5.0 again with SENTRY_EVENT_RETENTION_DAYS=3650 in .env, the events migrate from table sentry_local to table discover_local, but only those with 90 days back and so until the latest. Is there a way to keep all issues, it has also been tried to remove the sentry-cleanup cron as written in the README ("If you do not want the cleanup cron, you can remove the sentry-cleanup service from the docker-compose.yml file.") the result is the same - you see events only with 90 days back.

Expected Result

All issues are available and can be selected back through the WebUI, not just those from 90 days ago.

Actual Result

Only the issues from 90 days ago are visible in the WebUI.

Event ID

No response

Mandalavandalz commented 9 months ago

Also, after the update to the current latest (v24.1.1), with existing issues from 90 days ago and SENTRY_EVENT_RETENTION_DAYS=180 in .env, issues older than 90 days cannot be selected through the WebUI again. I let Sentry work for 2 days, and after 2 days, the oldest issues, which were visible before, cannot be selected during filtering from the WebUI with "Absolute date".

azaslavsky commented 9 months ago

Snuba team: I may be mistaken, but looking over https://github.com/search?q=repo%3Agetsentry%2Fsnuba%20DEFAULT_RETENTION_DAYS&type=code it seems like the DEFAULT_RETENTION_DAYS env variable is not actually ingested anywhere?

lojanp commented 3 months ago

can you please advise me? I have a self-hosted sentry running for about 40 days. I want to change settings for SENTRY_EVENT_RETENTION_DAYS from 90 days to 30 days in .env file. What steps do I need to take for the changes to take effect? I don't want to lose data. I use the docker compose solution. Well thank you.