datastax / starlight-for-kafka

DataStax - Starlight for Kafka
Apache License 2.0
17 stars 12 forks source link

[performance] Reduce the number of single byte writes to Direct ByteBufs #51

Closed eolivelli closed 1 year ago

eolivelli commented 1 year ago

From the flamegraph it is clear that the Kafka encoder calls too many times writeByte(byte) on DirectBufferOutputStream. This happens because we pass a DirectByteBuffer for the "value"

This is the code of Utils.writeTo()

public static void writeTo(DataOutput out, ByteBuffer buffer, int length) throws IOException {
        if (buffer.hasArray()) {
            out.write(buffer.array(), buffer.position() + buffer.arrayOffset(), length);
        } else {
            int pos = buffer.position();
            for (int i = pos; i < length + pos; i++)
                out.writeByte(buffer.get(i));
        }
    }
image

Modification:

eolivelli commented 1 year ago

superseded by #52