search

Zylann / godot_voxel

Voxel module for Godot Engine
MIT License
2.51k stars 236 forks source link

Blocky voxels revamp #506

Open Zylann opened 1 year ago

Zylann commented 1 year ago

This is the design of an upcoming change to how blocky voxels work.

The changes cover:

State combinations

In the current system, a new VoxelBlockyModel has to be made for every possible state voxels can have. So if we have a log type of voxel that can be oriented in 3 ways, then we have to make 3 pre-rotated models for each orientation:

If we have rails, we have to make a model for every orientations, turns and slopes as well.

In the new system, instead of directly adding VoxelBlockyModel resources to a VoxelBlockyLibrary, it expects VoxelBlockyType resources. A VoxelBlockyType can have one, or multiple VoxelBlockyModel, depending on states it can have.

The concept of "state" is inspired from how Minecraft works: each type of voxels can have "attributes" attached to them. The most common attributes are rotations, but plenty of others exist. When VoxelBlockyLibrary is baked, it will calculate all possible combinations of states a type can have from its attributes, and associate one model for each of them. If a type has rotations, then they can be generated automatically, but it may be possible to keep specifying models manually for best control.

States could be identified similar to Minecraft, using a name followed by a list of attributes (button[direction=+z,pressed=1]). This makes them easier to read, and possibly write in an in-game console or configuration files. This kind of notation is also what uniquely identifies a block state, including across mods. In the end, every game instance might have its own list of IDs for each block states, because it might use a different version of the game, with ot without mods.

Sources of inspiration from Minecraft: Website listing the attributes of every block: https://minecraftitemids.com/ Page from the Forge modding platform explaining block states: https://docs.minecraftforge.net/en/1.19.2/blocks/states/

Advantages of ID allocation

Under the hood, baking still works the same: each voxel value is associated to a unique, pre-rotated, ready-to-mesh model, with no need for complex conditions in the mesher. The setup is just no longer forced to be manual.

It is important to know that voxel values must be usable as indices in an array, because the mesher will perform thousands of lookups in order to produce a mesh from a chunk of voxels. If it was a HashMap, it would be slower. This is why it is important for states to be small and packed if possible.

Using multiple values for each state of a voxel type also makes a very efficient use of space. It saves memory and reduces the amount of data to send over network.

By default, each blocky voxel occupies 2 bytes, so there can be up to 65536 different states. If instead we reserved 5 bits to store up to 24 orthogonal rotations, it would limit the total amount of voxel types to 2048. It is a lot, but at the scale of Minecraft and when modding or extensions are included, it may not be enough. Many types of voxels might not even use those 5 bits, while the new system doesn't require that. Furthermore, some Minecraft blocks would need more than 5 bits to store their state (such as doors, fences and some redstone components).

Rotation attributes

Initially, the system will only offer rotation attributes. But the plan is to allow more, and some custom ones in the future.

Multiple types of rotations will be available:

The reason to have these different rotation attributes is space optimization. If a voxel type does not need rotation, or only need a few rotations, then it won't occupy as many possible values compared to a voxel type that can rotate in more ways. A log voxel type only needs 3 rotation states, while a button that can be placed on floors and ceilings will need at least 6. Meanwhile, dirt blocks will only use 1.

You could use FULL_ROTATION all the time for ease of use, but it would come at a higher cost. It would not only use more ID space, it would also occupy more memory to store all the pre-rotated variants. If all voxel types had this, we would run out of states after 2730 types (which is still more than if some bits were reserved).

Working with voxel values

In this system, voxel values are no longer "hardcoded" to some number that remains the same forever. It is generated to accomodate for each state, and perhaps even adapt to old saves. So in order to obtain the value of a voxel type, we will have to query it first using a function from VoxelBlockyLibrary.

In the new system, a type such as log can occupy 3 different voxel values at once in the voxel TYPE channel. So if we want to check if a voxel is of type log regardless of its state, we would have to check all 3 combinations. To make this easier, helper functions can be added to VoxelBlockyLibrary to convert between voxel values, types and states:

var state = voxel_tool.get_voxel(position, VoxelBuffer.CHANNEL_TYPE)
if voxel_library.get_type_name_from_state(state) == &"log":
    print("Found log")

Conversely if we want to place a vertical log in a script, we would do something like:

var state = voxel_library.get_state_id(&"log", [&"axis", VoxelBlockyAttributeAxis.Y])
voxel_tool.set_voxel(state, position, VoxelBuffer.CHANNEL_TYPE)

This is what the VoxelDemo project was doing as well, but it had to do it by hand using scripts, and it wasn't very clear. Note, if we do this in a tight loop placing many voxels, we can use a local variable to cache the state so it will save the cost of looking it up in every iteration.

API changes

Classes

Note: defining an AIR type might no longer be necessary, it would be generated automatically and always takes the value 0.

Backward-compatibility for Godot projects

If you have a VoxelBlockyLibrary in your project, the new system would break it because instead of an array of VoxelBlockyModel where indices matter, there would be an array of VoxelBlockyType instead where indices don't matter.

It might be possible to support old projects doing the following: When Godot loads a VoxelBlockyLibrary resource and that resource has the old voxel/* properties, the setter would recognize them and internally convert them into VoxelBlockyType resources, each of them having no attributes so they would correspond to the same states initially.

If you want to use the new system for rotations though, you will have to change your old setup.

Backward-compatibility for saved terrains

Supporting old saved terrains is trickier. This is a problem that existed even before introducing this system.

Baking a VoxelBlockyLibrary should produce the same IDs each time, unless types or attributes are modified.

Adding new voxel types is often not a problem, but removing or modifying them can cause old saves to load with janky results, since voxel values would no longer match the right states. Leaving IDs vacant leaving "holes" in the array of states somewhat solves the removal issue, but it means the space of possible values becomes polluted by old ones forever, which isn't very efficient. It also doesn't solve modifications of existing types, or changes in design that would require map conversions.

ID map

To solve this, saves may include what we could call an IDMap file. Such file would store a mapping between voxel states and their value in voxel data:

{
    "version": 1,
    "map": {
        "air": 0,
        "dirt": 1,
        "log[axis=x]": 1,
        "log[axis=x]": 2,
        "log[axis=x]": 3,
        "stone": 4,
        "grass": 5,
        "shrub1": 6
        "stairs[direction=-x]": 7,
        "stairs[direction=+x]": 8,
        "stairs[direction=-z]": 9,
        "stairs[direction=+z]": 10,
        "shrub2": 11
    }
}

If one day we decide to remove shrub1 from VoxelBlockyLibrary, if we bake that library, by default the generated IDs would end up like this:

{
    "version": 1,
    "map": {
        "air": 0,
        "dirt": 1,
        "log[axis=x]": 1,
        "log[axis=x]": 2,
        "log[axis=x]": 3,
        "stone": 4,
        "grass": 5,
        "stairs[direction=-x]": 6,
        "stairs[direction=+x]": 7,
        "stairs[direction=-z]": 8,
        "stairs[direction=+z]": 9,
        "shrub2": 10
    }
}

Notice how IDs shifted after grass. If we open the old save, stairs[direction=-x] would show up in every place where shrub1 used to be present.

So instead, what we can do is to provide the IDMap of the old save to VoxelBlockyLibrary, and bake it at runtime, so it will attempt to use existing IDs instead of generating a brand new series:

{
    "version": 1,
    "map": {
        "air": 0,
        "dirt": 1,
        "log[axis=x]": 1,
        "log[axis=x]": 2,
        "log[axis=x]": 3,
        "stone": 4,
        "grass": 5,

        "stairs[direction=-x]": 7,
        "stairs[direction=+x]": 8,
        "stairs[direction=-z]": 9,
        "stairs[direction=+z]": 10,
        "shrub2": 11
    }
}

This leaves a hole in the IDs, which the mesher will ignore as if it was air. Next time the game is saved, the updated IDMap will get saved too.

Such IDMap could also be present inside the VoxelBlockyLibrary itself, so such system could work without a saved file, as long as the resource has the data.

Conversion

But what if we want to replace the old shrub1 with shrub2 instead? What if we don't want holes polluting our IDs, and we want to use them for unrelated types later?

Also, if an IDMap has holes in it, adding new types may re-use these holes, which means old saves might suddenly have the new voxel type appear in coordinates that were storing an old removed type.

To handle this, we have to perform terrain conversion. Before loading a save, we could look at its IDMap file version, and compare it with the current version of the game. If they differ, then we have to convert the save. This may be as simple as going over all voxels and replace some values with other values.

{
    "version": 2,
    "migrations": {
        "shrub1": "shrub2"
    }
}

This tells the engine that to upgrade from version 1 to version 2, all coordinates containing the ID of shrub1 from the old IDMap must now contain the ID of shrub2 from the new IDMap.

Similarly, if one day we decide that log is no longer oriented using an axis, but has a direction instead, we could have this migration:

{
    "version": 3,
    "migrations": {
        "log[axis=x]": "log[direction=+x]",
        "log[axis=y]": "log[direction=+y]",
        "log[axis=z]": "log[direction=+z]",
    }
}

Note, because of this, values don't have to be contiguous over time, so it may not be exploited, unless we setup conversion to keep them packed as well.

Such conversions would have to be setup manually, because it is quite hard to automatically figure out how values match between old and new versions of the game, it's something only game design knows about.

Multiplayer

In order for clients and servers to communicate with the same IDs, they should share the same VoxelBlockyLibrary, or the same IDMap. This can be done either by shipping the same version of the game for client and server builds.

But if the game supports mods or extensions, the client may have to dynamically download the IDMap and VoxelBlockyLibrary from the server before it can load the map.

Yay or nay

If this is too many changes, it could be reduced down to the new VoxelBlockyModel* classes. VoxelBlockyLibrary would remain an array of models where indices matter. You would have to set them up manually, and handle all the things described above yourself using scripts.

Zylann commented 1 year ago

First batch of changes in the blocky_revamp branch:

image