search

owengage / fastnbt

Fast serde serializer and deserializer for Minecraft's NBT and Anvil formats
MIT License
185 stars 35 forks source link

Implement LZ4 decompression for 1.20.5+ #108

Closed Badel2 closed 3 months ago

Badel2 commented 4 months ago

Close #101

At first I tried using lz4_flex directly, but that doesn't work because java uses a different format. Then I searched "lz4 java" in crates.io and I found this nice crate that implements exactly what we need.

We need a Lz4DecoderWrapper because the API of that crate is not the same as the gzip crates. This introduces an additional copy but I believe the only way to avoid that would be to change the interface of the read_compressed_chunk function.

Tested by generating a world using a server with region-file-compression=lz4, and using the region-dump binary to test decompressing it.

owengage commented 4 months ago

Sorry I've not gotten around to looking at this yet.

From a brief look, my gut says I'd rather depend only on the lz4 dep directly and implement the layers on top directly. Mainly just to reduce dependencies. But that depends just how complex that would be.

Badel2 commented 4 months ago

Well I have no interest in learning about the implementation details of lz4, and copying that dependency as a new module would be pointless, so if you or someone else wants to work on that then nice, and if not then you can always use this pull request :)

No rush on my side as most worlds will not use this compression format yet.

owengage commented 3 months ago

Hey, I've still not got around to looking at this more closely. I'm going to go with this as is. I can always spend the time in future if I really care about the dependencies.

Thanks again for your contributions, it's appreciated.