Open errrov opened 4 months ago
The same errors occur in my kafka clients consuming messages by Consumer Group. The feature is that the usage rate of CPU of kafka client host is high and even up to 100% when the errors occur, which may be a reason causing Broken Pipe. In addition, the usage rate of CPU, memory or disk I/O of kafka server host seems well during these errors.
Description
Started getting errors of broken pipe and EOF when moved to go1.22. Maybe it coincided with changes on kafka server, want to get some information before passing this problem to company's kafka development team Consuming messages using Consumer Group. Want to understand when Broken Pipe error appear for Consumer Groups. When it sends offset to server? Can this problem be caused of consumer group code. Or its more likely kafka server side error?
Versions
Configuration
Logs
logs: CLICK ME
``` {"level":"error","error":"kafka: error while consuming testtopic/2: EOF","time":"2024-07-18T08:00:25+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/8: EOF","time":"2024-07-18T08:02:05+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/5: EOF","time":"2024-07-18T08:03:10+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/2: EOF","time":"2024-07-18T08:25:30+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/8: EOF","time":"2024-07-18T08:27:27+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/5: EOF","time":"2024-07-18T08:28:19+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/2: write tcp ip1->ip2: write: broken pipe","time":"2024-07-18T08:39:37+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/8: write tcp ip1->ip2: write: broken pipe","time":"2024-07-18T08:41:33+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/5: write tcp ip1->ip2: write: broken pipe","time":"2024-07-18T08:42:46+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/2: EOF","time":"2024-07-18T09:43:39+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/8: EOF","time":"2024-07-18T09:45:23+03:00","message":"Error occurred during consumer life-cycle"} {"level":"error","error":"kafka: error while consuming testtopic/5: EOF","time":"2024-07-18T09:46:25+03:00","message":"Error occurred during consumer life-cycle"} ```
Additional Context