Offroaders123 / NBTify

A library to read and write NBT files on the web!
http://npm.im/nbtify
MIT License
42 stars 5 forks source link

Little Endian Varint #34

Closed Offroaders123 closed 2 months ago

Offroaders123 commented 1 year ago

Going to add support for little endian varint encoding and decoding! This is inspired by the use-cases provided by Prismarine-NBT.

Bedrock Edition level format - wiki.vg janispritzkau / mcproto chrisdickinson / varint keithamus / varint CloudburstMC / NBT

Offroaders123 commented 1 year ago

Thanks to BJTMastermind Games for finding this article! I was wondering how Bedrock's Varint support differed to the ProtoBuf-like format itself, and it covers that very well. Extra helpful for setting this up, it even has an example of how to implement it!

VarInt And VarLong - wiki.vg

Pseudocode to read and write VarInts and VarLongs:

private static final int SEGMENT_BITS = 0x7F;
private static final int CONTINUE_BIT = 0x80;
public int readVarInt() {
    int value = 0;
    int position = 0;
    byte currentByte;

    while (true) {
        currentByte = readByte();
        value |= (currentByte & SEGMENT_BITS) << position;

        if ((currentByte & CONTINUE_BIT) == 0) break;

        position += 7;

        if (position >= 32) throw new RuntimeException("VarInt is too big");
    }

    return value;
}
public long readVarLong() {
    long value = 0;
    int position = 0;
    byte currentByte;

    while (true) {
        currentByte = readByte();
        value |= (long) (currentByte & SEGMENT_BITS) << position;

        if ((currentByte & CONTINUE_BIT) == 0) break;

        position += 7;

        if (position >= 64) throw new RuntimeException("VarLong is too big");
    }

    return value;
}
public void writeVarInt(int value) {
    while (true) {
        if ((value & ~SEGMENT_BITS) == 0) {
            writeByte(value);
            return;
        }

        writeByte((value & SEGMENT_BITS) | CONTINUE_BIT);

        // Note: >>> means that the sign bit is shifted with the rest of the number rather than being left alone
        value >>>= 7;
    }
}
public void writeVarLong(long value) {
    while (true) {
        if ((value & ~((long) SEGMENT_BITS)) == 0) {
            writeByte(value);
            return;
        }

        writeByte((value & SEGMENT_BITS) | CONTINUE_BIT);

        // Note: >>> means that the sign bit is shifted with the rest of the number rather than being left alone
        value >>>= 7;
    }
}
Offroaders123 commented 1 year ago

Think I may have found part of what I was missing here.

(May be notable for my newly-opened issue in Prismarine-NBT)

image image image
Offroaders123 commented 3 months ago

Progress for varint support moved just a little bit more forwards today! Debugging the current errors for things right now. Got help from GPT alongside the data on wiki.vg.

Varint Support Progress
Offroaders123 commented 2 months ago

NO WAY

Was listening to my debut album on Bandcamp, got to the breakdown with "💩💵", and then this started working. I love coincidences (intentional or not, sometimes things just work out it seems)

Varint finally works, wow!!
Offroaders123 commented 2 months ago

Ok, going to work on making a new npm release for everything in the last 6 months. While this isn't technically up yet when I close this, I'm closing this as a task in terms of it being feature-complete.