Given version 1.3.1, and a file of size between 100 and 799 bytes, with a cachePercent of 1.
When running
RandomAccessFile raf = new RandomAccessFile(new File("file799bytes.txt"), "r");
new FileWordList(raf, true, 1);
Then
java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:521)
at java.nio.ByteBufferAsLongBufferB.put(ByteBufferAsLongBufferB.java:128)
at org.passay.dictionary.AbstractFileWordList$Cache.put(AbstractFileWordList.java:386)
at org.passay.dictionary.AbstractFileWordList.initialize(AbstractFileWordList.java:139)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:138)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:109)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:83)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:62)
at org.passay.dictionary.FileWordList.<init>(FileWordList.java:43)
Analysis:
For small files that use a small cache, the cache buffer that is generated can be too small. This is due to how the cache is generated. In the constructor of cache the cacheSize is computed as:
final long cacheSize = (fileSize / 100) * cachePercent;
If this cacheSize is larger than 0, then used in resize(..) to create a ByteBuffer using:
ByteBuffer.allocate((int) size)
This ByteBuffer is then converted to a longBuffer using:
ByteBuffer.allocate((int) size).asLongBuffer()
Assume that the ByteBuffer is generated (cacheSize is larger than 0), then replacing the cacheSize formula in the ByteBuffer allocation, you get this:
The error happens when the ByteBuffer generated by allocate is less than 8 bytes, as asLongBuffer will then convert it to a long buffer of size 0 (allocateSize divided by 8, integer). When trying to putting values in an empty buffer, the BufferOverflowException is thrown.
Edit: It can also fail when the buffer is resized when calling cache.put(..).
A workaround is too disable the cache (cachePercent 0), use a cachePercent of at least 8, or not use small files. A fix would probably be to check for a too small cacheSize (< 8 ), and allocate at least 8 bytes in that case.
Given version 1.3.1, and a file of size between 100 and 799 bytes, with a cachePercent of 1.
When running
Then
Analysis:
For small files that use a small cache, the cache buffer that is generated can be too small. This is due to how the cache is generated. In the constructor of cache the cacheSize is computed as:
final long cacheSize = (fileSize / 100) * cachePercent;If this cacheSize is larger than 0, then used in
resize(..)to create a ByteBuffer using:ByteBuffer.allocate((int) size)This ByteBuffer is then converted to a longBuffer using:
ByteBuffer.allocate((int) size).asLongBuffer()Assume that the ByteBuffer is generated (cacheSize is larger than 0), then replacing the cacheSize formula in the ByteBuffer allocation, you get this:
ByteBuffer.allocate((int) (fileSize / 100) * cachePercent).asLongBuffer()Assume cachePercent is 1:
ByteBuffer.allocate((int) (fileSize / 100) * 1).asLongBuffer()ByteBuffer.allocate((int) (fileSize / 100)).asLongBuffer()The error happens when the ByteBuffer generated by
allocateis less than 8 bytes, asasLongBufferwill then convert it to a long buffer of size 0 (allocateSize divided by 8, integer). When trying to putting values in an empty buffer, the BufferOverflowException is thrown.For example, 100 bytes:
ByteBuffer.allocate((int) (100 / 100) * 1).asLongBuffer()ByteBuffer.allocate(1).asLongBuffer()1 size ByteBuffer -> asLongBuffer -> 0 size LongBufferFor example, 799 bytes:
ByteBuffer.allocate((int) (799 / 100) * 1).asLongBuffer()ByteBuffer.allocate(7).asLongBuffer()7 size ByteBuffer -> asLongBuffer -> 0 size LongBufferNo problem, 800 or more bytes:
ByteBuffer.allocate((int) (800 / 100) * 1).asLongBuffer()ByteBuffer.allocate(8).asLongBuffer()8 size ByteBuffer -> asLongBuffer -> 1 size LongBufferWhen using cachePercent of 2, it goes wrong up until 399 bytes:
ByteBuffer.allocate((int) (399 / 100) * 2).asLongBuffer()ByteBuffer.allocate(3 * 2).asLongBuffer()6 size ByteBuffer -> asLongBuffer -> 0 size LongBufferErrors (anything less than 8)
Edit: It can also fail when the buffer is resized when calling
cache.put(..).A workaround is too disable the cache (cachePercent 0), use a cachePercent of at least 8, or not use small files. A fix would probably be to check for a too small cacheSize (< 8 ), and allocate at least 8 bytes in that case.