Block model renderer: Propper post 'The Flattening' support

[Gyzie]

I'm just trying to get the ball rolling on this, but what I think Overviewer needs the most right now is a block model renderer. By that I mean that instead of defining in code how blocks should be rendered Overviewer would use the data defined in \assets\minecraft\models (from the .jar) to generate blocks.

So this would be that when Overviewer needs to render the block minecraft:brewing_stand it goes to the folder \assets\minecraft\blockstates and finds brewing_stand.json:

{
    "multipart": [
        {   "apply": { "model": "block/brewing_stand" }},
        {   "when": { "has_bottle_0": "true" },
            "apply": { "model": "block/brewing_stand_bottle0" }
        },
        {   "when": { "has_bottle_1": "true" },
            "apply": { "model": "block/brewing_stand_bottle1" }
        },
        {   "when": { "has_bottle_2": "true" },
            "apply": { "model": "block/brewing_stand_bottle2" }
        },
        {   "when": { "has_bottle_0": "false" },
            "apply": { "model": "block/brewing_stand_empty0" }
        },
        {   "when": { "has_bottle_1": "false" },
            "apply": { "model": "block/brewing_stand_empty1" }
        },
        {   "when": { "has_bottle_2": "false" },
            "apply": { "model": "block/brewing_stand_empty2" }
        }
    ]
}

_for now lets ignore the block states like has_bottle._ It sees that it's needs the model block/brewing_stand to render this block. With this information it goes to /assets/minecraft/models/block and finds brewing_stand.json:

{
    "textures": {
        "particle": "block/brewing_stand",
        "base": "block/brewing_stand_base",
        "stand": "block/brewing_stand"
    },
    "elements": [
        {   "from": [ 7, 0, 7 ],
            "to": [ 9, 14, 9 ],
            "faces": {
                "down":  { "uv": [ 7, 7, 9,  9 ], "texture": "#stand" },
                "up":    { "uv": [ 7, 7, 9,  9 ], "texture": "#stand" },
                "north": { "uv": [ 7, 2, 9, 16 ], "texture": "#stand" },
                "south": { "uv": [ 7, 2, 9, 16 ], "texture": "#stand" },
                "west":  { "uv": [ 7, 2, 9, 16 ], "texture": "#stand" },
                "east":  { "uv": [ 7, 2, 9, 16 ], "texture": "#stand" }
            }
        },
        {   "from": [ 9, 0, 5 ],
            "to": [ 15, 2, 11 ],
            "faces": {
                "down":  { "uv": [ 9,  5, 15, 11 ], "texture": "#base", "cullface": "down" },
                "up":    { "uv": [ 9,  5, 15, 11 ], "texture": "#base" },
                "north": { "uv": [ 9, 14, 15, 16 ], "texture": "#base" },
                "south": { "uv": [ 9, 14, 15, 16 ], "texture": "#base" },
                "west":  { "uv": [ 5, 14, 11, 16 ], "texture": "#base" },
                "east":  { "uv": [ 5, 14, 11, 16 ], "texture": "#base" }
            }
        },
        {   "from": [ 2, 0, 1 ],
            "to": [ 8, 2, 7 ],
            "faces": {
                "down":  { "uv": [ 2,  1, 8,  7 ], "texture": "#base", "cullface": "down" },
                "up":    { "uv": [ 2,  1, 8,  7 ], "texture": "#base" },
                "north": { "uv": [ 2, 14, 8, 16 ], "texture": "#base" },
                "south": { "uv": [ 2, 14, 8, 16 ], "texture": "#base" },
                "west":  { "uv": [ 1, 14, 7, 16 ], "texture": "#base" },
                "east":  { "uv": [ 1, 14, 7, 16 ], "texture": "#base" }
            }
        },
        {   "from": [ 2, 0, 9 ],
            "to": [ 8, 2, 15 ],
            "faces": {
                "down":  { "uv": [ 2,  9,  8, 15 ], "texture": "#base", "cullface": "down" },
                "up":    { "uv": [ 2,  9,  8, 15 ], "texture": "#base" },
                "north": { "uv": [ 2, 14,  8, 16 ], "texture": "#base" },
                "south": { "uv": [ 2, 14,  8, 16 ], "texture": "#base" },
                "west":  { "uv": [ 9, 14, 15, 16 ], "texture": "#base" },
                "east":  { "uv": [ 9, 14, 15, 16 ], "texture": "#base" }
            }
        }
    ]
}

With this .json file it can render the block in any given direction and apply the textures as defined in "textures".

(This is over simplified, but gives a general idea.)

The obvious benefit for this is that it's no longer necessary to manual define how blocks are rendered. The downside would be that Overviewer probably needs quite some work. But my knowledge about Overviewer nor Python or rendering isn't sufficient to do this on my own, but at the very least I could try to kickstart this.

So what I think we need to do:

• Drop support for pre 1.13 Minecraft versions: if you would like to render a pre 1.13 world you must use an older version of Overviewer.
• Identify what needs to be changed in Overviewer.
• Define and prioritise tasks.
• Write the new block renderer.

I think that there are some very passionate contributors to this project that would also love for this to be realised. So lets gather opinions and knowledge about doing this here. 👌

And ultimately realize it.

[IncredibleHolg]

Using a JSON description for the texture would be a big improvement, but there are some points to consider

• At the moment it is possible to use custom resourcepacks, containing only the PNG files. Unless we want to break this, we need to add the feature to stack resourcepacks (like the game does). Other wise you would get "JSON file not found" errors.
• The JSON files in the Minecraft JAR do not have any "Level Of Detail". Using them to render the scaled down images, which are needed for overviewer, creates unusable results.

What do you think of a adapted approach:

• create a block model renderer which supplied with JSON files.
• create adapted JSON files as part of the package

This would adress the LOD problem, and prevent resourcepacks fron breaking.

[Gyzie]

Alright good points! I appreciate it.

Could you explain more about your adapted approach? The reason why I would push for this block render is that it would make Overviewer support newer versions of Minecraft without any labour and as I understand right now you would like to create special block models in JSON for Overviewer? Which would required creating JSON files for every Minecraft update.

Regarding the LOD issue: I'm not too concerned about scaling down actual Minecraft blocks since they are already quite low on detail. As a quick test I tried scaling down 120px renders of blocks to 23px and found that it looks fine (or am I missing something?):

[Wunkolo]

Just wanted to comment that I am absolutely for this and it would totally lend itself easier to a GPU-accelerated backend I have in mind at some point and would totally be open to writing a software razterizer for the C backend. This would be great too as we can generate all the "sprites" up front and then just blit them around like we currently do but if we go in the direction of a 3d renderer then it would allow for a LOT more potential in terms of perspectives we can render at and overlays we can do. Also, by not just blitting raster sprites and dealing instead with 3d models then we can technically render at any resolution. So LOD is kind of a nonissue.

[DragonDev1906]

This would be really nice to have.

If we are planning to use OpenGL, I wan't to mention a possible Issue: This would require a Window Server (e.g. XServer) running on the executing server which might not be the case. One option around this could be EGL (https://devblogs.nvidia.com/egl-eye-opengl-visualization-without-x-server/).

About the Scaling issue: We could render the texture at 5 times the resolution and then downscale it to 24x24 pixel. That should still be visible from a distance and would be the same as the images scaled down.

How are we going to fix the "missing pixel issue" with the new Renderer? Copy the 6 pixels (maybe there is a better option by "overdrawing" and then cutting the image down to the needed pixels)?

For automatic ID generation we would also need to automatically generate the mapping to an Integer. I would suggest that every blockstate get's a Data ID assigned.

[iRath96]

I've just developed a software rasterizer that might be of interest for this: https://github.com/iRath96/block-rasterizer. It has no dependencies apart from numpy and PIL (which Minecraft-Overviewer already uses).

Texture generation performance (compared to Minecraft-Overviewer's current code) would be my main concern here. However, even though it won't be as fast as a GPU-based rasterizer, the implementation performs backface culling and uses numpy vectorization so it might just be efficient enough.

I fully agree with @Gyzie's recommendation to drop 1.13 support. The way Minecraft-Overviewer maps the palette to numeric ids would require a lot of hacks to be able to dynamically load models from JAR files.

I do not share the concerns of @IncredibleHolg:

• Stacking resource packs should be trivial (Minecraft-Overviewer already does this for textures anyway afaik)
• Level-of-detail regarding geometry should not really be an issue. Even blocks with fine structures (e.g. the brewing stand) can typically recognized in 24 pixel resolution. I don't even think supersampling would strictly be required as a pixelated look suits Minecraft quite well (but it's pretty simple to do if users desire it)

I'd like to note that we cannot automatically generate all blocks though: some blocks like chests or bells do not have full block models and instead rely on native rendering code.

[DragonDev1906]

I have implemented the texturegeneration using modernGL on https://github.com/DragonDev1906/Minecraft-Overviewer/tree/modernGLRender.

But additional requirements that were needed:

• modernGL
• Xwindow system (or similar), Windows or EGL if it is a headless server
• jar file must always be present (in addition to resource packs)
• Drop 1.13 support (although this is not strictly nessessary)

In addition some changes were made to the asset (texture) loading, a lot of code in textures.py needed to be changed (e.g. accessing textures/json files by name (minecraft:grass_block) instead of path (assets/minecraft/models/block/grass_block.json) and we needed a way to list/walk directories).

Unfortunately there were some problems / not yet implemented parts when integrating it into overviewer:

• which blocks have to be marked solid/transparent/...
• color tinting
• How to handle multipart blocks? Creating a texture for all possible combinations can result in over a million images (chorous flower). Ideally there would be a way for the C-code to tell the image generation if it needs a multipart image. Because of that multipart rendering is currently not implemented
• Apparently water doesn't work at the moment.

Btw: we might have fixed the calling generate() bug (e6dc4c2) but it was fixed after a lot of other changes.

As already mentioned some blocks can't be rendered because there is no data in the json (some can be partially rendered). One solution would be to create the json for those blocks in overviewer.

Unfortunately I have little time to continue at the moment but here are some images.

For the level of detail concern: Here are all blocks that can be rendered using a 24x24 area:

Here is one with a higher resolution (128x128 pixel):

And finally the result when integrated with overviewer:

[Wunkolo]

This looks absolutely great! I haven't looked at the texture generation code, but I would have personally gone with a Vulkan implementation over OpenGL simply because the API lends itself to headless rendering much easier and doesn't need a windowing system present at all to create images or framebuffers or render passes and such or require an alternative opengl implementation to be present. But would cull OSX support out of the benefits unless something like MoltenVK is used. An implementation-agnostic interface could be provided so that both a software renderer and an ogl/egl/vulkan/etc implementation can be selected.