search

apache / incubator-heron

Apache Heron (Incubating) is a realtime, distributed, fault-tolerant stream processing engine from Twitter
https://heron.apache.org/
Apache License 2.0
3.65k stars 597 forks source link

Gradual acking latency increase #1291

Open objmagic opened 8 years ago

objmagic commented 8 years ago

In PyHeron, acking latency increases gradually until it hits timeout. After that, all acking tuples will be dropped.

taishi8117 commented 8 years ago

We should investigate whether this issue still exists after the protobuf performance is improved.

objmagic commented 8 years ago

cf. #1217

kramasamy commented 8 years ago

@objmagic @taishi8117 - any update on this?

objmagic commented 8 years ago

not really sure how to start investigating this issue

objmagic commented 7 years ago

@kramasamy this should be the most important and the last issue. And I will investigate this issue.

objmagic commented 7 years ago

I am getting a bit lost when investigating this issue. Event loop could stop running, but not all the time. When I decrease parallelism to 1, acking latency (complete latency) is much slower, but still growing.