Querz / mcaselector

A tool to select chunks from Minecraft worlds for deletion or export.
MIT License
3.18k stars 177 forks source link

[feature] Hierarchical loading of chunk NBT #357

Open Mrkol opened 2 years ago

Mrkol commented 2 years ago

So uuuuh... I've made a flower farm in permanently loaded chunks, and it turned out to be efficient. No, you don't understand, it was very efficient. So now my server dies due to millions of item entities in those loaoded chunks. Yeah.

I've managed to fix this issue earlier by wiping the entity list from the chunk NBT using mcaselector, but this time is different. There are so many flowers, that mcaselector simply gives up while trying to parse the nbt.

Therefore my suggestion: make NBT loading hierarchical, i.e. don't try to parse the entire structure at once, only parse that which needs to be displayed. That would presumably allow me to open the chunk nbt, select the entity list without mcaselector parsing it and delete it.

I have not looked into how NBT format works internally, so it might be the case that hierarchical parsing is not possible. If that is the case, an alternative feature that would save us flower enjoyers would be a "delete all entities with id X" button. That would be even easier to implement, presumably.

Note that resetting the chunk would be very painful for me as it contains tons of modded entities and resources which may or may not count as multiblocks that would break and crash horribly due to chunk boundary shenanigans.

HoldYourWaffle commented 2 years ago

You might be able to resolve your situation with the more barebones NBT editor.

Based on the info on the wiki I'd say lazy-loading chunks of NBT would theoretically be possible, I might take a shot at it just for fun. Would it be possible for you to upload your world (or offending region files) somewhere for me test with?

Mrkol commented 2 years ago

No, NBT explorer does not support "external chunks", as in those that are too big and are stored separately from the region file in an auxiliary .mcс file.

As far as I can tell, mcaselector is the only editor style program that supports those files.

Here are the offending region and chunk files: https://drive.google.com/file/d/1kgOU1UsTcEE8lGrBkPvbbPX6pGD-JhPs/view?usp=sharing

HoldYourWaffle commented 2 years ago

Interesting, I've never encountered the mcc format before. It's nothing more than zlib-compressed NBT though, if you decompress it NBT explorer can edit it just fine :)

On the mcaselector side this issue seems to have been fixed already in the latest version, probably with the new NBT parser in 2.0. I was able to open your region file and read the chunks' NBT without any issues.

Querz commented 2 years ago

The OOM error happens because the NBT Editor runs out of memory when it tries to turn the entire NBT structure into tree items, not the file parsing.

I'll have a look at the NBT Editor code, i'm sure it can be improved so it only parses the visible tags into tree items.