1.18.1 Biome coloring are not applied

[Smoria]

Seems like biome coloring is not applied to any blocks if world is mc version 1.18 or later.

Using latest overviewer master produces next results:

As you can see there is no biome colors applied to grass/leaves and water. Snowy grass is also rendered as default green grass:

Config:

import os

worlds['minecraft'] = "/home/minecraft/server/world"
outputdir = "/home/minecraft/render/"

from .observer import MultiplexingObserver, LoggingObserver, JSObserver
loggingObserver = LoggingObserver()
jsObserver = JSObserver(outputdir)
observer = MultiplexingObserver(loggingObserver, jsObserver)

s_crop = (-5000, -5000, 5000, 5000)
s_center = [185, 64 -184]
s_zoom = 5

renders["day"] = {
    'world': 'minecraft',
    'title': 'Day',
    'rendermode': smooth-lighting,
    "dimension": "overworld",
    'crop': s_crop,
    'center': s_center,
    'defaultzoom': s_zoom
}

Minecraft v1.18.1 Overviewer 7171af5

[ST-Electro]

Same problem. For example, jungle and savanna colors missing:

worlds['Raksaservu'] = '/home/mc/server/Raksaservu'
outputdir = '/home/mc/server/overviewer/output'
texturepath = '/home/mc/server/overviewer/resourcepack'
customwebassets = '/home/mc/server/overviewer/web'
imgformat = 'jpeg'
imgquality = '75'

renders['normalrender'] = {
    'world': 'Raksaservu',
    'title': 'Normaali kartta',
    'rendermode': smooth_lighting,
    'northdirection': 'upper-right',
    'showlocationmarker':False,
    'poititle':'Karttamerkit'
}

Minecraft v1.18.1 Overviewer 0.17.47 (7171af5) Ubuntu Linux 18.04.6

[piegamesde]

This is very likely caused by changes in the save format in 1.18. As far as I can tell, the new biome encoding is not documented anywhere yet, so somebody would have to reverse it first.

[morsik]

@piegamesde looks like this project managed to get correct biome encding including biome colors to get working correctly with 1.18: https://unmined.net

just checked on my map, and it shows biome colors properly. Altough it's closed source (kinds weird decision in this ages since it's free anyway…) so I wonder where from author got docs about 1.18.

But file Stylesheets/Default/stylesheet.js (of this Unmined tool) gives clue about this biome coloring. Generally author just colors blocks for specific color based on biome name (but if I remember correctly, biome names were rearranged heavily in 1.18, so the current code in Overviewer may work, but it may need update regarding biome names…).

After some digging into Overviewer's code, I found out that it still uses IDs for biomes instead of new namespace text format which exists since many years… looks like only block were changed into namespaces :(

[piegamesde]

so I wonder where from author got docs about 1.18.

Probably directly from the Minecraft source code. I did the same (using the Fabric tutorial and example mod to get the decompiled+deobfuscated Minecraft) and apparently biomes are now encoded the same way as blocks, but with a different "resolution" for the palette. Blocks have "4 bits", biomes "2 bits". I did not dig deeper in their exact coordinate calculation, but 4 bits = 2⁴ = 16 and 2 bits = 2² = 4, which would make sense if interpreted as "resolution" because we already know that the biome resolution is 4x4x4 since the 1.16.

I've seen that chunkbase.com also has updated to 1.18 biome colors, and thus asked them to maybe provide the information (no response yet). Maybe we could contact the author of unmined as well?

I found out that it still uses IDs for biomes instead of new namespace text format which exists since many years

This makes sense, as the save format used the numeric ids until 1.18. Doing the migration with backwards compatibility in mind will be a pain though.

[morsik]

@piegamesde yeah, backwards compat topic was already mentioned by someone else few days ago in height limit PR: https://github.com/overviewer/Minecraft-Overviewer/pull/2012#issuecomment-1001638290

So maybe it's finally time to cut out 1.18 branch and leave rest as history in Overviewer… :D But that's decision to author of Overviewer.

[piegamesde]

Chunkbase was kind enough to provide the new colors (https://twitter.com/chunkbase/status/1480316017549131779):

[193, 165, 143], // dripstone
[223, 150, 52], // lush caves
[140, 164, 112], // meadow
[223, 236, 229], // grove
[218, 241, 241], // snowy slopes
[234, 251, 251], // frozen peaks
[227, 236, 237], // jagged peaks
[209, 209, 209], // stony peaks

Together with the extracted biome IDs from unmined (actually, some biome names in there were only up to 1.17 so I still had to do quite a bit of manual fixing), I combined it with my existing data to produce this file: https://github.com/Minecraft-Technik-Wiki/BlockMap/blob/12be724360a295dbdacb956301b907a8dd6ee5e2/BlockMap-internal/src/main/resources/biome-color-instructions.json

It contains enough information to extract everything you need for both the current and older versions. Actually, I'm thinking about moving the biome colors into their own repository so that we can all work on them together across tools.

[piegamesde]

I finally managed to reverse the biome storage format. It indeed is very similar to the blocks palette:

• Each palette entry requires n bits, according to the palette's size
• Those n bits are packed from low to high in a long, until no more fit. The rest of the long is padded with zeroes.
• Once the first long is full, the next long is filled, until there are no more palette entries left. The last long is filled with zeroes as well.
• Unlike the block palette, there is no minimum index of 4. Thus, for chunks with two biomes every bit is one biome, for chunks with three/four biomes every two bits are one biome, etc. This invites for some special case optimizations, especially since very few chunks have more than four biomes at once in them.

Java implementation (I'll put this into the NBT library some time later):

int paletteSize = palette.size();
/* This is just a rather weird integer log2 */
int bitsPerIndex = Integer.SIZE - Integer.numberOfLeadingZeros(paletteSize - 1);
int shortsPerLong = Math.floorDiv(64, bitsPerIndex);
int mask = (1 << bitsPerIndex) - 1;
/* 4×4×4 resolution within a chunk section (16×16×16). The usual ordering (XZY IIRC) */
String[] result = new String[64];
int index = 0;
for (long l : dataArray) {
    for (int i = 0; i < shortsPerLong && index < 64; i++) {
        result[index++] = palette.get((int) (l & mask));
        l >>= bitsPerIndex;
    }
}

[vilhok]

Is the biome color determined in the C or python code? I can't seem to find where that happens, though my C skills are almost nonexistent.

[CounterPillow]

Initial data munging is done in python: https://github.com/overviewer/Minecraft-Overviewer/blob/master/overviewer_core/biome.py lookup + coloring done in C: https://github.com/overviewer/Minecraft-Overviewer/blob/master/overviewer_core/src/primitives/base.c#L138

[duncy]

I am also having this issue in 1.19.1, I built from source. Do I need to clone a different branch instead of main?

Pictured is jungle, birch forest and savannah biomes all sharing the same leaf and grass colouring.

My config:

worlds["server"] = "/home/minecraft/server/world"

renders["normalrender"] = {
    "world": "server",
    "title": "overworld",
    "rendermode": smooth_lighting
}

outputdir = "/home/minecraft/map"

[Borous]

Encountering the same issue here, I'll apologize in advance as I'm terribly out of the loop with this nowadays, but has there been a solution found so far?

[vilhok]

@Borous I was discussing about this earlier in the thread. I've been tinkering around and I believe I've found the problem.

Everything seems to run fine until the biomes are applied and the result is that the get_data() call here returns always 0. The code is a bit tough to follow as the python code calls C-code which again calls pythoncode and so on (I've never worked on this or any project with this design). My best guess so far is that they are parsed in the wrong way and it seems that eventually the data and the biomes are parsed here: https://github.com/overviewer/Minecraft-Overviewer/blob/master/overviewer_core/nbt.py

I hope someone who is more familiar with the project could do the fix.

The guys at BlockMap -project seemingly solved this a year ago based on this commit. Following this might help with our implementation.

[CounterPillow]

The code seems quite clear on what is needed: https://github.com/overviewer/Minecraft-Overviewer/blob/6ffbe0f0beee56288fabce4db8d1838e42bac160/overviewer_core/world.py#L1708

Since Overviewer is unmaintained though, this work will not be done by me or anyone with write access to the repo.

[morsik]

@CounterPillow since you said "Overviewer is unmaintained" - is that means we should fork it? Or you'll accept PRs if someone will come with one?

[CounterPillow]

you should fork it, I'm not reviewing PRs anymore and I don't think anyone else is.