Create a voxel using godot_voxel

[acs]

In order to learn and explore godot_voxel, so we can use it later to render our MagicaVoxel models with voxels, the first step is to understand how a voxel is created in Godot using godot_voxel. So let's focus in this first step. Just adding a single voxel to a scene and render it.

After that, we will play wiith voxel properties and so on.

[acs]

First step, to compile the last version stable of Godot with the voxel module:

[godot](3.2.3-stable)$ git status .
HEAD detached at 3.2.3-stable
Untracked files:
  (use "git add <file>..." to include in what will be committed)
    modules/voxel
[godot](3.2.3-stable)$ scons -j8 platform=x11

[acs]

I will upload the development to:

https://github.com/acs/godot-samples/tree/master/vox2godot

[acs]

The new nodes added with godot_voxel are:

• VoxelLodTerrain
• VoxelTerrain
• VoxelViewer

Nothing about just a Voxel. Probably it should exist at some point. So we need to create the Voxel using the API provided by godot_voxel. This API is accesible from C++, GDScript and all the languages supported in Godot. It is documented at:

https://github.com/Zylann/godot_voxel/blob/master/doc/08_api-overview.md

After reading, the main API is for creating a terrain from a voxel data source. In our case, probably we should use Voxel to create a voxel (blocky) and then use the VoxelMesherBlocky to show it.

For example, we can add a MeshInstance to our scene and modify its mesh and assign to it the result of creating a VoxelMesherBlocky.

[acs]

We are pretty close: https://github.com/acs/godot-samples/blob/master/vox2godot/OneVoxel.gd

The code in https://github.com/Zylann/godot_voxel/issues/194#issuecomment-693708970 show the complete pipeline for importing MagicaVoxel models. But I would like to understand first how to create just one node.

[acs]

Ok, let's try to create a voxel terrain programatically and to understand how it is done. And then, simplify it until we reach just one voxel. Next step is to locate GDScript code to create terrain, not using the Nodes for Godot. Let's go!

It is simple to create a terrain with just one voxel, following https://github.com/Zylann/godot_voxel/blob/master/doc/06_custom-generator.md

But the idea is not to use the VoxelTerrain, but directly creating the mesh with the voxel to understand the internals.

[acs]

Ok, after talking with Zylann he provides me with a working version of the Vox model importer.

https://github.com/acs/godot-samples/tree/master/vox2godot

Now we can start analyzing the full pipeline!

The code can be read pretty well. The most important logic is the conversion of the voxels to a mesh to be visualized. This is done in a method that depends on the kind of mesher used.

In this case the meshed used is this build

[acs]

Dense voxels data storage.

Voxels (VoxelBuffer) are stored in dense structures.

[acs]

Ok, the code for loading a voxel model is understood. Now we must return to our goal of just loading 1 voxel.

[acs]

Time to create a model in MV with just 1 voxel:

Achieved. But using a process a bit complex. But this is how godot_voxel works:

• Voxels must be loaded in a VoxelBuffer
• Inside the buffer, voxels are placed in channels
• Each channel has properties that must be defined to create the mesh
• The voxels and the palette are loaded from the vox file using the VoxelVoxLoader
• voxels must be padded to include all of then in the mesh
• With the VoxelMesherCubes the voxels using the materials and the palette are rendered into a mesh
  • Material is needed to use the vertex colors as albedo
  • Palette is used to use the color defined in the voxel in MV
• The mesher is set to a MeshInstance so the mesh can be rendered
• For 1 cube 24 vertex are defined in the mesh (if exported from MV to obj and loaded in blender the mesh has 8 vertex)

Ok, the goal now is to create 1 voxel but without loading it from a vox file.

[acs]

The core logic for converting the voxels to a mesh is in the build_mesh method inside the mesher.

[acs]

Ok, let's try to complete this exercise without loading from a voxel model.

[acs]

I have seen in the project voxelgame, used for testing, that there is a kind of generator. And it creates a VoxelBuffer from scratch and fills it here: https://github.com/Zylann/voxelgame/blob/master/project/blocky_game/generator/test_generator.gd#L61

And this method shows different approaches for filling the voxel buffer: https://github.com/Zylann/voxelgame/blob/master/project/blocky_game/generator/generator.gd#L69

So let's study it with the goal of creating just 1 voxel. And easy task but not clearly document. And you should fill before some configs in the buffer for it to work.

[acs]

Ok, time to continue working with the generator. The goal is to understand how to create just one voxel. Once we understand it, we learnt the detail about how voxels are redered in godot_voxels.

This generator is based on the VoxelLibrary to create the meshes. And the method to create voxels with generate_block has more logic that the one used to load them from the vox file. So it is more complex than the previous effort of analyzing the loading of voxels from the vox file. So let's continue with it!

[acs]

Ok, let's go to the code and put the focus in understanding how a mesh is created from a VoxelBuffer. It is inside the C++ module code and make visible to GDScript here:

https://github.com/Zylann/godot_voxel/blob/master/meshers/voxel_mesher.cpp#L66

void VoxelMesher::_bind_methods() {
    // Shortcut if you want to generate a mesh directly from a fixed grid of voxels.
    // Useful for testing the different meshers.
    ClassDB::bind_method(D_METHOD("build_mesh", "voxel_buffer", "materials"), &VoxelMesher::build_mesh);
...

This is exactly what we want: a 1 grid voxel. Now we need tom understand what must be included in the mesher. For the vox importer is clear. But from a voxel created directly, not. Let's see how change:

        var _mesher = VoxelMesherCubes.new()
    var voxels = VoxelBuffer.new()
    voxels.set_channel_depth(VoxelBuffer.CHANNEL_COLOR, VoxelBuffer.DEPTH_8_BIT)
    var palette = VoxelColorPalette.new()
    var loader = VoxelVoxLoader.new()
    var err = loader.load_from_file("res://vox/1x1x1.vox", voxels, palette)
    _mesher.palette = palette
    _mesher.color_mode = VoxelMesherCubes.COLOR_MESHER_PALETTE
...
    var mesh = _mesher.build_mesh(padded_voxels, _materials)

for the equivalent but not loading the voxels, but just creating a single voxel like:

        var _mesher = VoxelMesherCubes.new()
    var voxels_raw = VoxelBuffer.new()
    voxels_raw.set_channel_depth(VoxelBuffer.CHANNEL_COLOR, VoxelBuffer.DEPTH_8_BIT)
    voxels_raw.create(1, 1, 1)
    voxels_raw.set_voxel(0, 0, 0, 0)
...
    var mesh = _mesher.build_mesh(padded_voxels, _materials)

The problem is that in the second case, the mesh is null, so the library can not create it using our 1 voxel VoxelBuffer created by hand. Let's recheck the reason.

The problem is debugging the build_mesh method because it is in C++. In this time let's just read it to understand what is doing.

Ok, until I have a better way to do it, I can print traces from C++ code:

ERROR: build_mesh: Test ERR_PRINT
   At: modules/voxel/meshers/voxel_mesher.cpp:7.
WARNING: build_mesh: Test WARN_PRINT
   At: modules/voxel/meshers/voxel_mesher.cpp:8

With our new powers it is easy to see that the problem is here:

if (voxels.get_channel_compression(channel) == VoxelBuffer::COMPRESSION_UNIFORM) {
        // All voxels have the same type.
        // If it's all air, nothing to do. If it's all cubes, nothing to do either.
        WARN_PRINT("Test WARN_PRINT VoxelMesherCubes build");

        return;

so in this case, a mesh is not created. And the problem is that the channel must be created, something done when loading the voxels from the vox file:

void VoxelBuffer::decompress_channel(unsigned int channel_index) {
    ERR_FAIL_INDEX(channel_index, MAX_CHANNELS);
    Channel &channel = _channels[channel_index];
    if (channel.data == nullptr) {
        create_channel(channel_index, _size, channel.defval);
    }
}

Is it possible to create a channel from GDScript? Let's see.

[acs]

To debug the C++ code from VSCode ... https://godotengine.org/qa/63553/how-to-debug-c-code-with-breakpoints-from-visual-studio This is my next step in order to learn more quickly all!

https://docs.godotengine.org/en/stable/development/cpp/configuring_an_ide/visual_studio_code.html

[acs]

Ok, as expected, all the problem goes with some missing step after the VoxelBuffer creation, related with setting the data in the channels. Now it is easy to spot it in the code. Now let's try to understand the logic.

[acs]

Ok, let's try to find doc about channels. It seems that they are properties about the VoxelBuffer but not sure if they have more things. In the Concepts section of godot_voxel https://github.com/Zylann/godot_voxel/blob/master/doc/08_api-overview.md#voxelbuffer. It seems that there is one channel per voxel.

VoxelBuffer

The underlying storage for voxels. Also used to pass voxels between functions.

    Stores an arbitrary number of voxels (Vector3int specifying how many in each direction).
    Stores up to 8 channels of arbitrary data per voxel. Each channel is allocated only upon use. (e.g. R,G,B,A, game play types (type, state, light), etc).

Inside VoxelBuffer when we create it, all the channels are created. Channels have:

• ChannelId (TYPE, SDF, COLOR, DATA3, ..., DATA7)
• Compression (NONE, UNIFORM)
• Depth: number of colors (8, 16, 32, 64)

When the VoxelBuffer is created:

        for (unsigned int i = 0; i < MAX_CHANNELS; ++i) {
            Channel &channel = _channels[i];
            if (channel.data) {
                // Channel already contained data
                delete_channel(i);
                create_channel(i, new_size, channel.defval);
            }
        }

Channels must be populated. This is done for example with the method set_voxel.

[acs]

Ok, I have got it:

    var voxels_raw = VoxelBuffer.new()
    voxels_raw.set_channel_depth(VoxelBuffer.CHANNEL_COLOR, VoxelBuffer.DEPTH_8_BIT)
    voxels_raw.create(1, 1, 1)
    voxels_raw.set_voxel(1, 0, 0, 0, VoxelBuffer.CHANNEL_COLOR)

The channel we need to set voxel to is the CHANNEL_COLOR. And the value must be >0. If it is 0, nothing is created because it is "air".

[acs]

Ok, so each channel has a data array field with the value for all the voxels in the VoxelBuffer.

Now I need to play with howto define the color of the voxel, but it should be easier.

Next steps:

• Understand better channels inside VoxelBuffer class
• Understand how colors and materials are managed
• Understand how a voxel is converted to a mesh (is it always a cube?). Explore the rest of the meshers.

[acs]

With the documentation about how the voxels data is persisted, we can find more information about channels!

https://github.com/Zylann/godot_voxel/blob/master/doc/specs/region_format_v3.md#channels

And reading the doc inside Godot for VoxelBuffer it says about the three first channels:

« enum ChannelId: CHANNEL_TYPE = 0 Channel used to store voxel types. Used by VoxelMesherBlocky.

CHANNEL_SDF = 1 Channel used to store SDF data (signed distance field). Used by VoxelMesherTransvoxel and other smooth meshers. Values should preferably be accessed as floats. Negative values are below the isosurface (inside matter), and positive values are above the surface (outside matter).

CHANNEL_COLOR = 2 Channel used to store color data. Used by VoxelMesherCubes. »

In our case we are using the VoxelMesherCubes so we must use the CHANNEL_COLOR to store the color value of the voxels. So everything is getting sense. Now we just need to understand how the colors and materials are managed.

There are other meshers which uses the other two channels to store the data. For example in the VoxelMesherBlocky the voxels are read from the CHANNEL_TYPE

void VoxelMesherBlocky::build(VoxelMesher::Output &output, const VoxelMesher::Input &input) {
    const int channel = VoxelBuffer::CHANNEL_TYPE;

In our case:

void VoxelMesherCubes::build(VoxelMesher::Output &output, const VoxelMesher::Input &input) {
    const int channel = VoxelBuffer::CHANNEL_COLOR;

and for the last mesher:

void VoxelMesherTransvoxel::build(VoxelMesher::Output &output, const VoxelMesher::Input &input) {

    int channel = VoxelBuffer::CHANNEL_SDF;

So we must store the data in the right channel so the mesher can find it. Great!

The format of the data inside the channel is defined by Compression and the Depth of the VoxelBuffer.

With the channel, compression and depth and a mesher, the mesh for the voxels is created!

[acs]

Ok, after thinking about it, with the VoxelMesherCubes the approach is to use a Palette to specify the full colors, and store the index to the color in the palette in the CHANNEL_COLOR data. But, could we use the palette from GDScript?

class VoxelColorPalette : public Resource {
    GDCLASS(VoxelColorPalette, Resource)

i think so. Let's try to use a palette in GDScript!

[acs]

Cool, I have it working and now I am pretty sure how all is working. In this example I add two voxels in x direction:

var _mesher = VoxelMesherCubes.new()
var _palette = VoxelColorPalette.new()

func _load_palette():
    _mesher.palette = _palette
    # The first entry in the palette must be Color8(0,0,0,0)
    _mesher.palette.set_color(0, Color8(0,0,0,0))
    _mesher.palette.set_color(1, Color8(238,0,0))
    _mesher.palette.set_color(2, Color8(0,238,0))

func _ready():
    _load_palette()
    var voxels = VoxelBuffer.new()
    voxels.set_channel_depth(VoxelBuffer.CHANNEL_COLOR, VoxelBuffer.DEPTH_8_BIT)
    voxels.create(2, 1, 1)
    voxels.set_voxel(2, 0, 0, 0, VoxelBuffer.CHANNEL_COLOR)
    voxels.set_voxel(1, 1, 0, 0, VoxelBuffer.CHANNEL_COLOR)

Ok, so just missing howto play with the materials. But this is something not needed now. So I think we can close this issue.