Events getting skipped

[VatsalLalcheta]

Checklist

• [x] I have included information about relevant versions
• [x] I have verified that the issue persists when using the master branch of Faust.

Steps to reproduce

I am using aws msk cluster for running kafka and I have multiple faust agents listening to different topics in my service. I have noticed sometimes, some events are getting skipped and not consumed by the faust agents. After that, I have to restart my service in order to process those events. This should never happen. Also, it doesn't show any lag in offset. So, probably it is storing these events in some buffer topic or something. What could be the possible reason for this?

Expected behavior

All the events that are being produced to the topic should get consumed by the agent.

Actual behavior

Some intermittent events are getting skipped by the faust agent. It won't even show any error log or anything. It doesn't show any consumer offset lag.

Full traceback

Versions

• Python version - 3.9
• Faust version - 1.10.4
• Operating system
• Kafka version
• RocksDB version (if applicable)

[dada-engineer]

According to the version you mention this looks like you are using the abandoned Robinhood Faust lib. Please make sure you are using faust-streaming. Latest version is 0.10.22.

If the issue persists please feel free to reach out with a code snippet to see what you are doing.