search

lz4 / lz4-java

LZ4 compression for Java
Apache License 2.0
1.09k stars 248 forks source link

support bytebuffer updates on streaming hashes #187

Open pyr opened 2 years ago

pyr commented 2 years ago

In some cases ByteBuffers do not provide access to an underlying byte array without incurring copies, this is notably the case for off-heap direct bytebuffers.

This patch provides a way to call update on streaming XXHASH instances against a ByteBuffer, which avoids the extra copy needed into a byte array.

I'm putting this one up to gauge interest. If present, I can adapt the Java safe and unsafe variants to also support the method, I first wanted some feedback on the general idea.

I'm also a bit unclear on what to do should we reach a case where the buffer supports neither .isDirect nor .hasArray, in this case it feels as though the only way forward is to allocate a byte array to copy the ByteBuffer contents into

pyr commented 9 months ago

Hi, a quick update on this one, it has been running fine for over a year in production. Is there any interest in seeing this get in or should I close the PR?