search

logstash-plugins / logstash-input-couchdb_changes

This plugin captures the _changes stream from a CouchDB instance
Apache License 2.0
27 stars 22 forks source link

Update speed degrades over time #11

Closed konradkonrad closed 6 years ago

konradkonrad commented 9 years ago

Hi there! I am using v0.1.4 with logstash 1.5.0rc2 for ~10 days now.

My configuration:

In the first 1-2 days, the small db (overall ~15mio seq events) got up-to-date, big (~250mio seq events) synced around ~45mio seq events.

edit: so speed was at least ~30,000 seq events per minute during that period.

I did not monitor the progress closely after the first 2 days. When I checked back today, small was still up-to-date, but the update speed for big was down to only ~1000 events per minute, overall it was only at ~60mio.

The logstash agent is started with --debug, but there was nothing in the log/output. The elasticsearch target (5 data-nodes) seemed stable (load < 0.5, HEAP "okay").

After restarting logstash, the updating speed went up to ~200,000 seq events per minute.

untergeek commented 6 years ago

Closing this as it's so old, with no updates.