Having a way to identify problematic message and the possibility to remove those messages from the journal - or at least log if the processing takes to much time.
Current Behavior
Large messages can slow down or block the processing. Currently there is no way to find out which message is the reason for that. Currently you can only wait until the messages are processed, building up a large backlog of message. Or delete the journal to remove the problematic messages (and lose messages).
Possible Solution
After a configured limit of processing time is reach for a single message, log a message to the server.log including details like journal offset, message id etc.
Optional cancel the processing for that message after a timeout to be able to process the backlog.
lennartkoopmann
commented
7 years ago
Expected Behavior
Having a way to identify problematic message and the possibility to remove those messages from the journal - or at least log if the processing takes to much time.
Current Behavior
Large messages can slow down or block the processing. Currently there is no way to find out which message is the reason for that. Currently you can only wait until the messages are processed, building up a large backlog of message. Or delete the journal to remove the problematic messages (and lose messages).
Possible Solution
After a configured limit of processing time is reach for a single message, log a message to the server.log including details like journal offset, message id etc.
Optional cancel the processing for that message after a timeout to be able to process the backlog.