Add dedupe on server side for sentry

[Sammaye]

Problem Statement

Sentry has a few tools to stop spikes from flooding the server such as throttle, sdk dedupe and few others but it doesn't have a truly useful mechanism that has no data loss.

I recently noticed this issue as I drained the errors on my plan and got locked out of the account due to small mistake in my code that was called many times by an external service.

I was advised that I would have to make something myself in my application but that kind of defeats the point of an error handler since what if that middle-ware throws an error? There is a high chance of data loss rendering sentry pointless if I did that.

Solution Brainstorm

I propose that Sentry should have a mechanism to drop errors should they match a fingerprint+fields over a time, for example: if the same error comes in with the same fingerprint and url and userId within a 5 minute window then drop the error.

This would create a dynamic throttle that avoids data loss.

[getsantry[bot]]

Assigning to @getsentry/support for routing, due by Friday, March 3rd at 5:00 pm (sfo). ⏲️

[getsantry[bot]]

Routing to @getsentry/team-web-sdk-backend for triage, due by Tuesday, March 7th at 10:48 am (sfo). ⏲️

[sl0thentr0py]

@JLuse this is a product feature request, not sure which team to route to

[getsantry[bot]]

Routing to @getsentry/ingest for triage, due by Wednesday, March 8th at 15:34 (vie). ⏲️

[souredoutlook]

Thanks @sl0thentr0py

@Sammaye I've re-routed this to our Ingest team since it's related to server-side filtering. But I wanted to let you know I raised it internally with another team as well that's responsible for Issue Relevance. Hopefully we'll get the right eyes on this.

[hubertdeng123]

Putting this on the Backlog so that this issue doesn't get tracked for Triage, since it's a feature request