search

buger / goreplay

GoReplay is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data. It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.
https://goreplay.org
Other
18.46k stars 9 forks source link

send kafka error #1168

Open zer0e opened 1 year ago

zer0e commented 1 year ago

[DEBUG][elapsed 6.2756408s]: Failed to write access log entry: kafka: Failed to produce message to topic gor-1679038301199: kafka server: Message was too large, server rejected it to avoid allocation error.

I set kafka server config max.message.bytes=104857600, but goreplay is also send error. Should I config other settings?

zer0e commented 1 year ago

Well, the kafka producer should add c.Producer.MaxMessageBytes config to enable the message size larger 1M. If possible, goreplay can add flag to set MaxMessageBytes with KafkaOutput?