Sentry stops accepting errors

[barisyild]

Self-Hosted Version

24.8.0

CPU Architecture

x86_64

Docker Version

27.2.1

Docker Compose Version

2.29.2

Steps to Reproduce

-

Expected Result

Sentry should have accepting errors.

Actual Result

About every 24 hours the sentry stops accepting errors. The problem is not solved without restarting Sentry.

relay-1 | 2024-09-14T10:22:17.355619Z DEBUG request{method=POST uri=/api/10/envelope/ version=HTTP/1.1}: tower_http::trace::on_request: started processing request
relay-1 | 2024-09-14T10:22:17.356611Z ERROR request{method=POST uri=/api/10/envelope/ version=HTTP/1.1}: relay_server::endpoints::common: error handling request error=failed to queue envelope
relay-1 | 2024-09-14T10:22:17.356735Z DEBUG request{method=POST uri=/api/10/envelope/ version=HTTP/1.1}: tower_http::trace::on_response: finished processing request latency=1 ms status=503
relay-1 | 2024-09-14T10:22:17.356755Z DEBUG request{method=POST uri=/api/10/envelope/ version=HTTP/1.1}: tower_http::trace::on_failure: response failed classification=Status code: 503 Service Unavailable latency=1 ms
relay-1 | 2024-09-14T10:22:17.357352Z ERROR relay_server::services::health_check: Not enough memory, 16071245824 / 16372436992 (98.16% >= 95.00%)
relay-1 | 2024-09-14T10:22:17.358510Z ERROR relay_server::services::health_check: Health check probe 'system memory' failed

System has 16 GB ram, swap memory disabled. I can send full logs from pm.

Event ID

No response

[bijancot]

Hi, just take a peek at this issue, what abaout the web container? is the logs look strange?

In my case this issue happened once and never happen again, but the web seems not having enough sockets to open end up custom some config in web container and the redis it self

[hubertdeng123]

How often is this error happening? What does your event volume look like?

[bijancot]

I'll try to get it here soon. Unfortunately we don't have uptime monitoring for our sentry because it counts as internal tools. But it's likely happen almost every day back then, I'll try to undo our changes we made and get the logs

Chers !

Pada Rab, 18 Sep 2024 pukul 06.15 Hubert Deng @.***> menulis:

How often is this error happening? What does your event volume look like?

— Reply to this email directly, view it on GitHub https://github.com/getsentry/self-hosted/issues/3327#issuecomment-2357146879, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE5HC7VROF7IVOFLLMH3B2TZXCZ2BAVCNFSM6AAAAABOGXFPNGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJXGE2DMOBXHE . You are receiving this because you commented.Message ID: @.***>

[aldy505]

For this specific warn on your logs:

relay-1 | 2024-09-14T10:22:17.357352Z ERROR relay_server::services::health_check: Not enough memory, 16071245824 / 16372436992 (98.16% >= 95.00%)
relay-1 | 2024-09-14T10:22:17.358510Z ERROR relay_server::services::health_check: Health check probe 'system memory' failed

Please refer to this https://github.com/getsentry/self-hosted/issues/3330#issuecomment-2364791946

[getsantry[bot]]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀