Retrieving indices from Index 255 and creating client cache requests

Going through this library and attempting to use it for a while, I still have yet to find a way to generate a response for the client when the client is loading up and rends a request to the server for the cache archives/files. Also, even when successfully loading the cache, there is no archive inside the getIndex255() object to retrieve those archives for that cache request either.

What revision are you working with?

What revision are you working with?

666

I didn't think it mattered because it loads the cache entirely, and I can still retrieve data with no errors.

This is my current way to generate a response

/**
     * Generates a response for the client's file request.
     * 
     * @param idx    the idx of the file
     * @param fileId the file id corresponding to the file
     * @param opcode
     * @return
     * @throws Exception
     */
    public ByteBuffer loadResponse(IDX idx, int fileID, boolean addAttributes) throws Exception {
        ByteBuffer out, raw;
        int length, compression;

        if (idx == IDX.META && fileID == 255) {
            raw = checksum.encode(true);
            compression = 0;
            length = raw.remaining();
        } else {
            raw = ByteBuffer.wrap(getFileData(idx, fileID));
            compression = raw.get() & 0xFF;
            length = raw.getInt();
        }

        out = ByteBuffer.allocate(raw.remaining() + 8 + ((raw.remaining() + 8) / 512) + 4);
        int attribs = compression;
        if (addAttributes)
            attribs |= 0x80;
        out.put((byte) idx.getArchiveIndex());
        out.putShort((short) fileID);
        out.put((byte) attribs);
        out.putInt(length);
        raw.limit(raw.position() + length + (compression == 0 ? 0 : 4));

        while (raw.remaining() > 0) {
            if (out.position() % 512 == 0)
                out.put((byte) 0xFF);
            out.put(raw.get());
        }
        out.flip();
        return out;
    }

Here's the one I am using in 562, it uses the old API, meaning it was always there:

public class UpdateThread extends Thread {

    /**
     * The lock object.
     */
    private static final Lock LOCK = new ReentrantLock();

    @Override
    public void run() {
        while(true) {
            try {
                final RequestEntry request = Update.DISPATCH_QUEUE.next();
                if (request != null) {
                    if (request.getIndex() == 255 && request.getContainer() == 255) {
                        final int length = CacheRepository.getChecksumTable().getData().length;
                        writeChunked(request, CacheRepository.getChecksumTable().getData(), 0, 0, length, length);
                    } else {
                        final byte[] data = (request.getIndex() == 255 ? CacheRepository.getChecksumTable().getArchiveInformation(request.getContainer()) : CacheRepository.getIndices()[request.getIndex()].getArchiveInformation(request.getContainer())).getData();
                        final int compression = data[0] & 0xFF;
                        int settings = compression;
                        if (request.getPriority() == RequestPriority.HIGH) {
                            settings |= 0x80;
                        }
                        final int length = ((data[1] & 0xff) << 24) + ((data[2] & 0xff) << 16) + ((data[3] & 0xff) << 8) + (data[4] & 0xff);
                        writeChunked(request, data, compression, settings, length, compression != 0 ? length + 4 : length);
                        if (request.getPriority() == RequestPriority.LOW) {
                            System.out.println("Requested container=" + request);
                        }
                    }
                }
            } catch (InterruptedException e) {
                e.printStackTrace();
                continue;
            }
            try {
                Thread.sleep(10);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }

    /**
     * Write the update request in chunks of 512 bytes to the communication channel.
     * @param request The request reference.
     * @param data The data byte buffer.
     * @param compression The compression.
     * @param length The length in chunks of 512 bytes.
     */
    private static void writeChunked(RequestEntry request, byte[] data, int compression, int settings, int length, int realLength) {
        ByteBuf buffer = Unpooled.buffer();
        buffer.writeByte(request.getIndex());
        buffer.writeShort(request.getContainer());
        buffer.writeByte(settings);
        buffer.writeInt(length);
        if (request.getIndex() == 255 && request.getContainer() == 255) {
            int offset = 10;
            for (int index = 0; index < realLength; index++) {
                if (offset == 512) {
                    buffer.writeByte(255);
                    offset = 1;
                }
                buffer.writeByte(data[index]);
                offset++;
            }
        } else {
            for (int i = 5; i < realLength + 5; i++) {
                if (buffer.writerIndex() % 512 == 0) {
                    buffer.writeByte(255);
                }
                buffer.writeByte(data[i]);
            }
            if (request.getEncryption() != 0) {
                for (int i = 0; i < buffer.writerIndex(); i++) {
                    buffer.setByte(i, buffer.getByte(i) ^ request.getEncryption());
                }
            }
        }
        write(request.getChannel(), buffer);
    }

    /**
     * Write the buffer to the communication channel.
     * @param channel The channel.
     * @param buffer The buffer.
     */
    private static void write(Channel channel, ByteBuf buffer) {
        LOCK.lock();
        if (channel.isActive()) {
            channel.writeAndFlush(buffer);
        }
        LOCK.unlock();
    }

}

Here's what changed:

cache.getChecksumTable() has been renamed to cache.index255

However, I simplified this even more. I think for 666 you need the new ukeys format. You can replace

cache.getChecksumTable().getData()

with

cache.generateNewUkeys(your_exponent, your_modulus)

For old revisions like 562 (I forgot when they changed this) you need:

cache.generateOldUkeys()

the getArchiveInformation(id) has been renamed to readArchiveSector(id)

Ah, I've never needed to get an exponent or modulus when generating the whirlpool. Would you be able to add an option not to use exponent/modulus on the buffer.writeBytes(Buffer.cryptRSA(whirlpoolBuffer.array(), exponent, modulus)) in the CacheLibrary class? Instead, just return the whirlpool buffer. Unless you have any tips on how to define the RSA values.

I doubt you want that. There are only 2 versions of the so called ukeys. There is the old one which does not need an exponent and modulus and one which does. You should have either one, unless you edited that part in your client. What does your checksum.encode(true) method look like? Compare it with the generateOldUkeys and generateNewUkeys from the cache library to see which one you need.

This is actually Graham's from Rune-Server ChecksumTable that I'm using. I believe it's from OpenRS. I understand how all of this works, but I've never worked with a client that needed an encrypted block of code using RSA. To be fair though, I've only used 637 and 666 over the past couple years.

    public ByteBuffer encode(boolean whirlpool) throws IOException {
        return encode(whirlpool, null, null);
    }

    /**
     * Encodes this {@link ChecksumTable} and encrypts the final whirlpool hash.
     * 
     * @param whirlpool  If whirlpool digests should be encoded.
     * @param modulus    The modulus.
     * @param privateKey The private key.
     * @return The encoded {@link ByteBuffer}.
     * @throws IOException if an I/O error occurs.
     */
    public ByteBuffer encode(boolean whirlpool, BigInteger modulus, BigInteger privateKey) throws IOException {
        ByteArrayOutputStream bout = new ByteArrayOutputStream();
        DataOutputStream os = new DataOutputStream(bout);
        try {
            /*
             * as the new whirlpool format is more complicated we must write the number of
             * entries
             */
            if (whirlpool)
                os.write(entries.length);

            /* encode the individual entries */
            for (int i = 0; i < entries.length; i++) {
                Entry entry = entries[i];
                os.writeInt(entry.crc());
                os.writeInt(entry.version());
                if (whirlpool)
                    os.write(entry.whirlpool());
            }

            /* compute (and encrypt) the digest of the whole table */
            if (whirlpool) {
                byte[] bytes = bout.toByteArray();
                ByteBuffer temp = ByteBuffer.allocate(66);
                temp.put((byte) 0);
                temp.put(Whirlpool.whirlpool(bytes, 5, bytes.length - 5));
                // temp.put((byte) 0);
                temp.flip();

                if (modulus != null && privateKey != null) {
                    temp = crypt(temp, modulus, privateKey);
                }

                bytes = new byte[temp.limit()];
                temp.get(bytes);
                os.write(bytes);
            }

            byte[] bytes = bout.toByteArray();
            return ByteBuffer.wrap(bytes);
        } finally {
            os.close();
        }
    }

Ah, it looks like there is one more version that I am missing then, which is the old version with whirlpool support. I never used my library in 562 - 718 so I guess I missed that one, will add it tomorrow and make a new release.

You're awesome and responsive!

Any updates? I only know a little kotlin, but not enough to contribute to the source :/

Here is the whirlpool that you can use. This can also replace your current whirlpool class you have.

package com.displee.util;

import java.util.Arrays;

/**
 * The Whirlpool hashing function.
 *
 * <P>
 * <b>References</b>
 *
 * <P>
 * The Whirlpool algorithm was developed by
 * <a href="mailto:pbarreto@scopus.com.br">Paulo S. L. M. Barreto</a> and
 * <a href="mailto:vincent.rijmen@cryptomathic.com">Vincent Rijmen</a>.
 *
 * See P.S.L.M. Barreto, V. Rijmen, ``The Whirlpool hashing function,'' First
 * NESSIE workshop, 2000 (tweaked version, 2003),
 * <https://www.cosic.esat.kuleuven.ac.be/nessie/workshop/submissions/whirlpool.zip>
 *
 * @author Paulo S.L.M. Barreto
 * @author Vincent Rijmen.
 *
 * @version 3.0 (2003.03.12)
 *
 *          =============================================================================
 *
 *          Differences from version 2.1:
 *
 *          - Suboptimal diffusion matrix replaced by cir(1, 1, 4, 1, 8, 5, 2,
 *          9).
 *
 *          =============================================================================
 *
 *          Differences from version 2.0:
 *
 *          - Generation of ISO/IEC 10118-3 test vectors. - Bug fix: nonzero
 *          carry was ignored when tallying the data length (this bug apparently
 *          only manifested itself when feeding data in pieces rather than in a
 *          single chunk at once).
 *
 *          Differences from version 1.0:
 *
 *          - Original S-box replaced by the tweaked, hardware-efficient
 *          version.
 *
 *          =============================================================================
 *
 *          THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS
 *          OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 *          WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 *          ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE
 *          LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 *          CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 *          SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
 *          BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 *          LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 *          NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 *          SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 *
 */
public class Whirlpool {

    /**
     * The message digest size (in bits)
     */
    public static final int DIGESTBITS = 512;

    /**
     * The message digest size (in bytes)
     */
    public static final int DIGESTBYTES = DIGESTBITS >>> 3;

    /**
     * The number of rounds of the internal dedicated block cipher.
     */
    protected static final int R = 10;

    /**
     * The substitution box.
     */
    private static final String sbox = "\u1823\uc6E8\u87B8\u014F\u36A6\ud2F5\u796F\u9152"
            + "\u60Bc\u9B8E\uA30c\u7B35\u1dE0\ud7c2\u2E4B\uFE57" + "\u1577\u37E5\u9FF0\u4AdA\u58c9\u290A\uB1A0\u6B85"
            + "\uBd5d\u10F4\ucB3E\u0567\uE427\u418B\uA77d\u95d8" + "\uFBEE\u7c66\udd17\u479E\ucA2d\uBF07\uAd5A\u8333"
            + "\u6302\uAA71\uc819\u49d9\uF2E3\u5B88\u9A26\u32B0" + "\uE90F\ud580\uBEcd\u3448\uFF7A\u905F\u2068\u1AAE"
            + "\uB454\u9322\u64F1\u7312\u4008\uc3Ec\udBA1\u8d3d" + "\u9700\ucF2B\u7682\ud61B\uB5AF\u6A50\u45F3\u30EF"
            + "\u3F55\uA2EA\u65BA\u2Fc0\udE1c\uFd4d\u9275\u068A" + "\uB2E6\u0E1F\u62d4\uA896\uF9c5\u2559\u8472\u394c"
            + "\u5E78\u388c\ud1A5\uE261\uB321\u9c1E\u43c7\uFc04" + "\u5199\u6d0d\uFAdF\u7E24\u3BAB\ucE11\u8F4E\uB7EB"
            + "\u3c81\u94F7\uB913\u2cd3\uE76E\uc403\u5644\u7FA9" + "\u2ABB\uc153\udc0B\u9d6c\u3174\uF646\uAc89\u14E1"
            + "\u163A\u6909\u70B6\ud0Ed\ucc42\u98A4\u285c\uF886";

    private static long[][] C = new long[8][256];
    private static long[] rc = new long[R + 1];

    static {
        for (int x = 0; x < 256; x++) {
            char c = sbox.charAt(x / 2);
            long v1 = ((x & 1) == 0) ? c >>> 8 : c & 0xff;
            long v2 = v1 << 1;
            if (v2 >= 0x100L) {
                v2 ^= 0x11dL;
            }
            long v4 = v2 << 1;
            if (v4 >= 0x100L) {
                v4 ^= 0x11dL;
            }
            long v5 = v4 ^ v1;
            long v8 = v4 << 1;
            if (v8 >= 0x100L) {
                v8 ^= 0x11dL;
            }
            long v9 = v8 ^ v1;
            /*
             * build the circulant table C[0][x] = S[x].[1, 1, 4, 1, 8, 5, 2, 9]:
             */
            C[0][x] = (v1 << 56) | (v1 << 48) | (v4 << 40) | (v1 << 32) | (v8 << 24) | (v5 << 16) | (v2 << 8) | (v9);
            /*
             * build the remaining circulant tables C[t][x] = C[0][x] rotr t
             */
            for (int t = 1; t < 8; t++) {
                C[t][x] = (C[t - 1][x] >>> 8) | ((C[t - 1][x] << 56));
            }
        }

        /*
         * build the round constants:
         */
        rc[0] = 0L; /* not used (assigment kept only to properly initialize all variables) */
        for (int r = 1; r <= R; r++) {
            int i = 8 * (r - 1);
            rc[r] = (C[0][i] & 0xff00000000000000L) ^ (C[1][i + 1] & 0x00ff000000000000L)
                    ^ (C[2][i + 2] & 0x0000ff0000000000L) ^ (C[3][i + 3] & 0x000000ff00000000L)
                    ^ (C[4][i + 4] & 0x00000000ff000000L) ^ (C[5][i + 5] & 0x0000000000ff0000L)
                    ^ (C[6][i + 6] & 0x000000000000ff00L) ^ (C[7][i + 7] & 0x00000000000000ffL);
        }
    }

    /**
     * Global number of hashed bits (256-bit counter).
     */
    protected byte[] bitLength = new byte[32];

    /**
     * Buffer of data to hash.
     */
    protected byte[] buffer = new byte[64];

    /**
     * Current number of bits on the buffer.
     */
    protected int bufferBits = 0;

    /**
     * Current (possibly incomplete) byte slot on the buffer.
     */
    protected int bufferPos = 0;

    /**
     * The hashing state.
     */
    protected long[] hash = new long[8];
    protected long[] K = new long[8]; // the round key
    protected long[] L = new long[8];
    protected long[] block = new long[8]; // mu(buffer)
    protected long[] state = new long[8]; // the cipher state

    public Whirlpool() {
    }

    public static byte[] whirlpool(byte[] data, int off, int len) {
        byte source[];
        if (off <= 0) {
            source = data;
        } else {
            source = new byte[len];
            for (int i = 0; i < len; i++)
                source[i] = data[off + i];
        }
        Whirlpool whirlpool = new Whirlpool();
        whirlpool.NESSIEinit();
        whirlpool.NESSIEadd(source, len * 8);
        byte digest[] = new byte[64];
        whirlpool.NESSIEfinalize(digest);
        return digest;
    }

    /**
     * The core Whirlpool transform.
     */
    protected void processBuffer() {
        /*
         * map the buffer to a block:
         */
        for (int i = 0, j = 0; i < 8; i++, j += 8) {
            block[i] = (((long) buffer[j]) << 56) ^ (((long) buffer[j + 1] & 0xffL) << 48)
                    ^ (((long) buffer[j + 2] & 0xffL) << 40) ^ (((long) buffer[j + 3] & 0xffL) << 32)
                    ^ (((long) buffer[j + 4] & 0xffL) << 24) ^ (((long) buffer[j + 5] & 0xffL) << 16)
                    ^ (((long) buffer[j + 6] & 0xffL) << 8) ^ (((long) buffer[j + 7] & 0xffL));
        }
        /*
         * compute and apply K^0 to the cipher state:
         */
        for (int i = 0; i < 8; i++) {
            state[i] = block[i] ^ (K[i] = hash[i]);
        }
        /*
         * iterate over all rounds:
         */
        for (int r = 1; r <= R; r++) {
            /*
             * compute K^r from K^{r-1}:
             */
            for (int i = 0; i < 8; i++) {
                L[i] = 0L;
                for (int t = 0, s = 56; t < 8; t++, s -= 8) {
                    L[i] ^= C[t][(int) (K[(i - t) & 7] >>> s) & 0xff];
                }
            }
            for (int i = 0; i < 8; i++) {
                K[i] = L[i];
            }
            K[0] ^= rc[r];
            /*
             * apply the r-th round transformation:
             */
            for (int i = 0; i < 8; i++) {
                L[i] = K[i];
                for (int t = 0, s = 56; t < 8; t++, s -= 8) {
                    L[i] ^= C[t][(int) (state[(i - t) & 7] >>> s) & 0xff];
                }
            }
            for (int i = 0; i < 8; i++) {
                state[i] = L[i];
            }
        }
        /*
         * apply the Miyaguchi-Preneel compression function:
         */
        for (int i = 0; i < 8; i++) {
            hash[i] ^= state[i] ^ block[i];
        }
    }

    /**
     * Initialize the hashing state.
     */
    public void NESSIEinit() {
        Arrays.fill(bitLength, (byte) 0);
        bufferBits = bufferPos = 0;
        buffer[0] = 0; // it's only necessary to cleanup buffer[bufferPos].
        Arrays.fill(hash, 0L); // initial value
    }

    /**
     * Delivers input data to the hashing algorithm.
     *
     * @param source     plaintext data to hash.
     * @param sourceBits how many bits of plaintext to process.
     *
     *                   This method maintains the invariant: bufferBits < 512
     */
    public void NESSIEadd(byte[] source, long sourceBits) {
        /*
         * sourcePos | +-------+-------+------- ||||||||||||||||||||| source
         * +-------+-------+------- +-------+-------+-------+-------+-------+-------
         * |||||||||||||||||||||| buffer
         * +-------+-------+-------+-------+-------+------- | bufferPos
         */
        int sourcePos = 0; // index of leftmost source byte containing data (1 to 8 bits).
        int sourceGap = (8 - ((int) sourceBits & 7)) & 7; // space on source[sourcePos].
        int bufferRem = bufferBits & 7; // occupied bits on buffer[bufferPos].
        int b;
        // tally the length of the added data:
        long value = sourceBits;
        for (int i = 31, carry = 0; i >= 0; i--) {
            carry += (bitLength[i] & 0xff) + ((int) value & 0xff);
            bitLength[i] = (byte) carry;
            carry >>>= 8;
            value >>>= 8;
        }
        // process data in chunks of 8 bits:
        while (sourceBits > 8) { // at least source[sourcePos] and source[sourcePos+1] contain data.
            // take a byte from the source:
            b = ((source[sourcePos] << sourceGap) & 0xff) | ((source[sourcePos + 1] & 0xff) >>> (8 - sourceGap));
            if (b < 0 || b >= 256) {
                throw new RuntimeException("LOGIC ERROR");
            }
            // process this byte:
            buffer[bufferPos++] |= b >>> bufferRem;
            bufferBits += 8 - bufferRem; // bufferBits = 8*bufferPos;
            if (bufferBits == 512) {
                // process data block:
                processBuffer();
                // reset buffer:
                bufferBits = bufferPos = 0;
            }
            buffer[bufferPos] = (byte) ((b << (8 - bufferRem)) & 0xff);
            bufferBits += bufferRem;
            // proceed to remaining data:
            sourceBits -= 8;
            sourcePos++;
        }
        // now 0 <= sourceBits <= 8;
        // furthermore, all data (if any is left) is in source[sourcePos].
        if (sourceBits > 0) {
            b = (source[sourcePos] << sourceGap) & 0xff; // bits are left-justified on b.
            // process the remaining bits:
            buffer[bufferPos] |= b >>> bufferRem;
        } else {
            b = 0;
        }
        if (bufferRem + sourceBits < 8) {
            // all remaining data fits on buffer[bufferPos], and there still remains some
            // space.
            bufferBits += sourceBits;
        } else {
            // buffer[bufferPos] is full:
            bufferPos++;
            bufferBits += 8 - bufferRem; // bufferBits = 8*bufferPos;
            sourceBits -= 8 - bufferRem;
            // now 0 <= sourceBits < 8; furthermore, all data is in source[sourcePos].
            if (bufferBits == 512) {
                // process data block:
                processBuffer();
                // reset buffer:
                bufferBits = bufferPos = 0;
            }
            buffer[bufferPos] = (byte) ((b << (8 - bufferRem)) & 0xff);
            bufferBits += (int) sourceBits;
        }
    }

    /**
     * Get the hash value from the hashing state.
     *
     * This method uses the invariant: bufferBits < 512
     */
    public void NESSIEfinalize(byte[] digest) {
        // append a '1'-bit:
        buffer[bufferPos] |= 0x80 >>> (bufferBits & 7);
        bufferPos++; // all remaining bits on the current byte are set to zero.
        // pad with zero bits to complete 512N + 256 bits:
        if (bufferPos > 32) {
            while (bufferPos < 64) {
                buffer[bufferPos++] = 0;
            }
            // process data block:
            processBuffer();
            // reset buffer:
            bufferPos = 0;
        }
        while (bufferPos < 32) {
            buffer[bufferPos++] = 0;
        }
        // append bit length of hashed data:
        System.arraycopy(bitLength, 0, buffer, 32, 32);
        // process data block:
        processBuffer();
        // return the completed message digest:
        for (int i = 0, j = 0; i < 8; i++, j += 8) {
            long h = hash[i];
            digest[j] = (byte) (h >>> 56);
            digest[j + 1] = (byte) (h >>> 48);
            digest[j + 2] = (byte) (h >>> 40);
            digest[j + 3] = (byte) (h >>> 32);
            digest[j + 4] = (byte) (h >>> 24);
            digest[j + 5] = (byte) (h >>> 16);
            digest[j + 6] = (byte) (h >>> 8);
            digest[j + 7] = (byte) (h);
        }
    }

    /**
     * Delivers string input data to the hashing algorithm.
     *
     * @param source plaintext data to hash (ASCII text string).
     *
     *               This method maintains the invariant: bufferBits < 512
     */
    public void NESSIEadd(String source) {
        if (source.length() > 0) {
            byte[] data = new byte[source.length()];
            for (int i = 0; i < source.length(); i++) {
                data[i] = (byte) source.charAt(i);
            }
            NESSIEadd(data, 8 * data.length);
        }
    }

}

Here is a method to generate the new u keys for revision 637-666 i believe. I don't know other revisions that use this. Maybe 614 to 718? Who knows...

    fun generateNewUkeys(whirlpool: Boolean = false): ByteArray {
        val bout = ByteArrayOutputStream()
        val os = DataOutputStream(bout)
        if (whirlpool) {
            os.write(indices.size)
        }

        for (index in indices()) {
            os.writeInt(index.crc)
            os.writeInt(index.revision)
            if (whirlpool)
                os.write(index.whirlpool)
        }
        if (whirlpool) {
            var bytes = bout.toByteArray()
            val temp = ByteBuffer.allocate(66)
            temp.put(0)
            temp.put(Whirlpool.whirlpool(bytes, 5, bytes.size - 5))
            temp.flip()
            bytes = ByteArray(temp.limit())
            temp.get(bytes)
            os.write(bytes)
        }
        return bout.toByteArray()
    }

Sorry, had a busy week. I've drafted a new release:

implementation("com.displee:rs-cache-library:6.9-RC1")

Use the following function:

generateUkeys()

Could you test this and let me know if it works? I've tested it on a 634 so far and it seems to work.

On 666, this does not work. I'm sure it's just missing something small. The UKeys length should be 2730 for 666, but your method provided only generates a length of 2670. Gotta go to work, but I'll compare the differences after

Okay. Lets try it one more time (repo available in 10 minutes or so):

implementation("com.displee:rs-cache-library:6.9-RC2")

Use the following function:

generateUkeys2()

Okay. Lets try it one more time (repo available in 10 minutes or so):
implementation("com.displee:rs-cache-library:6.9-RC2")
Use the following function:
generateUkeys2()

Works wonderfully. You're awesome Displee, no doubt about it. I'm scraping my cache library for yours, it loads faster, but also the memory usage is less with yours for some reason. Makes me happy. Hopefully people other than me will benefit from this in the future.

If so, here is how I do the response generation:

private final ByteBuffer generateResponse(IDX idx, int fileID, boolean addAttributes) throws Exception {
    ByteBuffer out, raw;
    int length, compression;

    if (idx == IDX.META && fileID == 255) {
        raw = ByteBuffer.wrap(Network.CACHE.generateUkeys2(true));
        compression = 0;
        length = raw.remaining();
    } else {
        Index index = idx == IDX.META ? Network.CACHE.getIndex255() : Network.CACHE.index(idx.getIndex());
        raw = ByteBuffer.wrap(index.readArchiveSector(fileID).getData());
        compression = raw.get() & 0xFF;
        length = raw.getInt();
    }

    out = ByteBuffer.allocate(raw.remaining() + 8 + ((raw.remaining() + 8) / 512) + 4);
    out.put((byte) idx.getIndex());
    out.putShort((short) fileID);
    out.put((byte) (addAttributes ? compression | 0x80 : compression));
    out.putInt(length);
    raw.limit(raw.position() + length + (compression == 0 ? 0 : 4));

    while (raw.remaining() > 0) {
        if (out.position() % 512 == 0)
            out.put((byte) 0xFF);
        out.put(raw.get());
    }
    out.flip();
    return out;
}

Displee / rs-cache-library

Retrieving indices from Index 255 and creating client cache requests #46