Schematic file format for 1.13 and newer is not supported

[erich666]

Short version: there's a new .schem file format, made by volunteers. Mineways does not support it. If you want to render such files, simply import them into an application that does read such data and save to a standard Mojang-supported world file, which Mineways does read. Update: the Litematica tool is evidently good for reading such .schem files and inserting them into worlds.

Evidently FAWE supports output of a new schematic file format for their system for Minecraft: https://www.spigotmc.org/resources/fast-async-worldedit-voxelsniper.13932/

Unfortunately, this format is not documented in any place I could find. The traditional, pre-1.13 schematic format is documented here: https://minecraft.gamepedia.com/Schematic_file_format

That said, I suspect someone could go through the code starting here in FAWE and figure out what they do.

The new and old are considerably different, since 1.13 and pre-1.13 are considerably different. The pre-1.13 format looks like this:

It's pretty simple: XYZ dimensions, then the block and data fields in huge arrays.

The new format is more involved:

It doesn't look all that bad, until you poke around.

There's some Metadata - not sure about what that is, but probably easy to figure out. The Palette is also fairly obvious, but with a huge number of options to parse:

The TileEntities also look "interesting":

BlockData is confusing. It is an array of size 30,518,469 But the dimensions Height Length Width, 230 436 304, multiply to 30,485,120 - a difference of 33,349. Why?

Basically, this looks like a lot of work to read in, having to parse and convert all possible Palette entries is a fair bit of mindless work. So, I'm passing for now, as there are more interesting things in the world to do than laborious palette parsing for a mod's feature. However, I will add code to see if a schematic format being read in is this new format and, if so, pop up a warning.

[erich666]

Added code to issue this error on read:

I also now issue a warning if you try to export a 1.13 world and a 1.13 block type is encountered:

[IronApollo]

Hello. I'm currently the lead (and in all honesty, the only) developer of FAWE. Things have been slow and I'm tied up IRL, but I'm here.

The schematic format supported by FAWE and WE prior to 1.13 was the MCEDIT schematic format. Due to the changes in the way Minecraft handles, saves and reads blocks starting in 1.13, WE and FAWE have moved on from the MCEDIT schematic format to the Sponge schematic format. This format is more suitable and ideal for the 1.13 environment, and hopefully for updates to come. This is a change that likely will not be reverted, and as for FAWE, we no longer support any writing feature related to the MCEDIT schematic format, but reading is still possible.

If you are interested in knowing more about the new default schematic format, you can Google "Sponge schematic format". I would provide a link, but I'm mobile at the moment.

If you have any further questions regarding FAWE and the way it handles schematics, feel free to message me on Discord @IronApollo#1953, or join the IntellectualSites Discord server to engage in conversation on the topic.

[erich666]

Josh, thanks for your reply, and I'm impressed you found this issue at all. Like I say, adding new schematic read support for 1.13 is low on my list; so far only one user has reported needing it. But, I'm happy to know where to look for more information: https://github.com/SpongePowered/Schematic-Specification/blob/master/versions/schematic-1.md

I'll contact you on Discord - I'm definitely interested in other elements of FAWE, such as being able to read "non-classic" Minecraft files formats the way MCEdit does.

[erich666]

@IronApollo Hmmm, unfortunately, my attempts to find you or IntellectualSites on Discord have failed. I'd certainly appreciate it if you contact me directly, I'm here. It'd be nice to talk with someone else supporting reading Minecraft files; I promise I won't spam you.

[craftyfoxe]

@erich666 Hi, I'm actually the one who asked IronApollo to help with you since you were having a problem and I think if Mineways supported 1.13 schematics that would be awesome to render stuff.

You can come to the Intellectual Discord server by going to this link: https://discord.gg/ebreupz

[acs]

Hi all,

I have added Schematic write and write support for https://github.com/juntosdesdecasa/mcthings. But using the legacy format for MCEdit, because the amount of creations in this format is larger and because I am working with the Raspberry Pi API (for Python programming) which uses the old block ids definition before the Minecraft flattening: https://minecraft.gamepedia.com/Java_Edition_1.13/Flattening.

I am pretty sure @erich666 that to be future proof, Mineways will need to support the new way to define the blocks in Minecraft, so the move to the support for the Sponge schematic format seems to be a must to use Mineways with all the new Schematics creations (and as time goes by, there will be more Schematics in this new format).

@erich666 thanks for your great tool which is the perfect bridge between Minecraft and the 3D rendering and printing world!

Just my 2 cents!

[erich666]

Thanks for your kind words, they mean a lot to me. Unfortunately, updating this feature is a ton of really boring work. But, I'll think about it - boring might be just the thing to keep my mind off the rest of the world. I might figure out some good way to leverage the code that's there.

Duplicate report, see https://github.com/erich666/Mineways/issues/40, so I'm closing it. But, thanks for writing, as it led me to the Sponge Format documentation (I didn't know that was what it was called until today).

[erich666]

Oh, wups, you put this info in the right issue! Reopened. The Sponge specification is here: https://github.com/SpongePowered/Schematic-Specification

[acs]

@erich666 the conversion between blocks ids and materials is already done in some places (like for example, Spigot adds a Legacy Material layer if it finds old plugins loaded that use the old block ids to specify blocks). I have research it a bit in https://github.com/juntosdesdecasa/mcthings/issues/58 and this Legacy support is included in Craftbukkit:

https://hub.spigotmc.org/stash/projects/SPIGOT/repos/craftbukkit/browse/src/main/java/org/bukkit/craftbukkit/legacy/CraftLegacy.java

But it seems it depends in Java libraries from the vanilla Minecraft server:

https://github.com/Akarin-project/Minecraft/blob/master/1.13.2/spigot/net/minecraft/server/DataConverterFlattenData.java

I don't know how WorldEdit support also both version, but probably it must be analyzed before doing the work of supporting the materials specification and its mapping to block ids.

The mapping is also documented in https://minecraft.gamepedia.com/Java_Edition_1.13/Flattening#Block_and_Item_IDs so probably this seems to be the best source of info. Parsing this HTML to generate the mapping should not be too difficult.

And there is also this list from a bug in Mojang with a txt file describing the mapping:

https://bugs.mojang.com/secure/attachment/151784/the_flattening.txt

I hope it helps!

[erich666]

@IronApollo @craftyfoxe @acs I could use your help. Do any of you know of a utility that will read in the new-fangled schematic format you've made and convert it/insert it into a Java (Classic) Minecraft world? The utility needs to not be a mod, like WorldEdit.

I have a user of Mineways that wants to read in a schematic. My normal advice is "read it into a world," but they say "the schematic file has flowers floating, if you put it in the world it will be damaged", so the WorldEdit mod can't be used. So, is there another application they could use?

[IronApollo]

@IronApollo @craftyfoxe @acs I could use your help. Do any of you know of a utility that will read in the new-fangled schematic format you've made and convert it/insert it into a Java (Classic) Minecraft world? The utility needs to not be a mod, like WorldEdit.

I have a user of Mineways that wants to read in a schematic. My normal advice is "read it into a world," but they say "the schematic file has flowers floating, if you put it in the world it will be damaged", so the WorldEdit mod can't be used. So, is there another application they could use?

Hey erich,

I'm not sure which schematic format you're referring to (I assume .schem aka Sponge Schematic Format). If it is the Sponge Schematic Format you're referring to, it has become the new standard in the world modification scene and I would highly recommend implementing proper support if you find the time!

It's been a while since I have worked on FAWE, but at the time I stopped working on it, this would be a great case to use it as FAWE did not impose block updates on edits as a way to increase world modification speeds. You mention not using a mod, so I don't have a solution for you unless they are willing to paste it into the world using FAWE and see if it keeps their floating flowers. There are important distinctions to be made between FAWE and WE with a lack of block updates in FAWE being one of the largest.

Hope this helps.

[erich666]

@IronApollo yes, I meant your new Sponge Schematic Format. And, thanks, I will point them at FAWE.

[erich666]

One other way to manipulate .schem files is to use Litematica, recommended by a person on Reddit.