search

sentry-extensions / cyclops

Cyclops is meant to be a high performance barrier in front of Sentry (http://getsentry.com).
http://sentry-extensions.github.io/cyclops/
164 stars 15 forks source link

With Cyclops, is Celery needed? #7

Closed Rylon closed 10 years ago

Rylon commented 10 years ago

Hey there,

We're using Sentry with Celery in front, and I wondered if the recommended way to deploy Cyclops was without Celery in between, or is it perfectly fine to configure things so that Cyclops -> Celery -> Sentry?

Thanks!

heynemann commented 10 years ago

Not at all... we use cyclops here at globo.com with just a regular running sentry (gunicorn). Right now we have this processing stats (with two servers running cyclops in front of two servers running sentry):

firstserver.domain.com:9100 has still 0 messages to process
firstserver.domain.com:9101 has still 0 messages to process
firstserver.domain.com:9102 has still 0 messages to process
firstserver.domain.com:9103 has still 0 messages to process
firstserver.domain.com:9104 has still 0 messages to process
firstserver.domain.com:9105 has still 0 messages to process
firstserver.domain.com:9106 has still 0 messages to process
firstserver.domain.com:9107 has still 0 messages to process
firstserver.domain.com:9108 has still 0 messages to process
firstserver.domain.com:9109 has still 0 messages to process
firstserver.domain.com:9110 has still 0 messages to process
firstserver.domain.com:9111 has still 0 messages to process
firstserver.domain.com:9112 has still 0 messages to process
firstserver.domain.com:9113 has still 0 messages to process
firstserver.domain.com:9114 has still 0 messages to process
firstserver.domain.com:9115 has still 0 messages to process
firstserver.domain.com:9116 has still 0 messages to process
firstserver.domain.com:9117 has still 0 messages to process
firstserver.domain.com:9118 has still 0 messages to process
firstserver.domain.com:9119 has still 0 messages to process
firstserver.domain.com:9120 has still 0 messages to process
firstserver.domain.com:9121 has still 0 messages to process
firstserver.domain.com:9122 has still 0 messages to process

Total of 0 messages to send to sentry from the farm at firstserver.domain.com.

Total 3732016 processed items and 84491884 ignored items (95.77%).
Average sentry response time is 119.63ms and 90% Percentile is 103.49ms

secondserver.domain.com:9100 has still 0 messages to process
secondserver.domain.com:9101 has still 0 messages to process
secondserver.domain.com:9102 has still 0 messages to process
secondserver.domain.com:9103 has still 0 messages to process
secondserver.domain.com:9104 has still 0 messages to process
secondserver.domain.com:9105 has still 0 messages to process
secondserver.domain.com:9106 has still 0 messages to process
secondserver.domain.com:9107 has still 0 messages to process
secondserver.domain.com:9108 has still 0 messages to process
secondserver.domain.com:9109 has still 0 messages to process
secondserver.domain.com:9110 has still 0 messages to process
secondserver.domain.com:9111 has still 0 messages to process
secondserver.domain.com:9112 has still 0 messages to process
secondserver.domain.com:9113 has still 0 messages to process
secondserver.domain.com:9114 has still 0 messages to process
secondserver.domain.com:9115 has still 0 messages to process
secondserver.domain.com:9116 has still 0 messages to process
secondserver.domain.com:9117 has still 0 messages to process
secondserver.domain.com:9118 has still 0 messages to process
secondserver.domain.com:9119 has still 0 messages to process
secondserver.domain.com:9120 has still 0 messages to process
secondserver.domain.com:9121 has still 0 messages to process
secondserver.domain.com:9122 has still 0 messages to process

Total of 0 messages to send to sentry from the farm at secondserver.domain.com.

Total 3728142 processed items and 84496168 ignored items (95.77%).
Average sentry response time is 119.31ms and 90% Percentile is 103.42ms

These results are obtained with the commands:

@cyclops-count --host=firstserver.domain.com --ports=9100-9122
@cyclops-count --host=secondserver.domain.com --ports=9100-9122

Hope that helps! :)

Rylon commented 10 years ago

Fantastic, thanks :)

One other quick question, what time period do these events cover? ie/ ~3.7 million requests since when?

heynemann commented 10 years ago

Since my last message, the first server has processed 3,742,988 items and ignored 84,710,072 items. Subtracting from previous gives 10,972 processed items and 218,188 ignored items.

Summing both gives 229,160 items. Since I commented around 2 hours ago, that would be 7200 seconds. Dividing one by the other gives around 32 items per second per server (give or take).

Does that help?

Rylon commented 10 years ago

Yeah, very helpful, thanks! :+1: