From the flamegraph it is clear that the Kafka encoder calls too many times writeByte(byte) on DirectBufferOutputStream.
This happens because we pass a DirectByteBuffer for the "value"
This is the code of Utils.writeTo()
public static void writeTo(DataOutput out, ByteBuffer buffer, int length) throws IOException {
if (buffer.hasArray()) {
out.write(buffer.array(), buffer.position() + buffer.arrayOffset(), length);
} else {
int pos = buffer.position();
for (int i = pos; i < length + pos; i++)
out.writeByte(buffer.get(i));
}
}
Modification:
introduce a simple buffer in order to group the writes to direct memory
From the flamegraph it is clear that the Kafka encoder calls too many times
writeByte(byte)
on DirectBufferOutputStream. This happens because we pass a DirectByteBuffer for the "value"This is the code of Utils.writeTo()
Modification: