Support for complexe shape

[SirJackblue]

Hello !

Recently, I started to work on a Voxel Engine, and your project helped me a lot ! I would like to add support for complexe shape, not only draw brick/voxel. I understand how the Brick Render Component works but I have few questions concerning:

• How unreal normalize UByte4N, normally each byte will be normalized with this formula: divide to 255 = x / max_x * 255, where max_x is the highest byte value and x the current byte value. My question here is does Unreal follow this rule.
• How uint8 X is converted to UByte4N (I think each bytes has the value of X) ?
• What is the usage of the 4th bytes of the UByte4N ?
• If I use a Float4 instead of UByte4N for the ComponentPosition of the DataType, and I normalize it like a UByte4N, can I achieve what I want (complexe bricks like stairs, half-block, etc..) ?

Thank you in advance :)

Cordially, Jackblue

[iUltimateLP]

Personally, I'd use the Procedural Mesh Component for geometry which doesn't belong to the terrain. Example: I used the Procedural Mesh Component to create runtime-loaded meshes which I can place in the world (select a FBX file, it parses the fbx (using assimp) and gets the information out and populates a procedural mesh with it ). If you want custom shapes for the terrain, I'd take a look at @miguelemosreverte's branch, he implemented wedges on the terrain :smile:

[SirJackblue]

Thank you for your reply @iUltimateLP !

I prefer to use the terrain grid, for performance and replication purpose, because those shapes I want to draw can be contained inside a brick shape. If I need to render a bigger model, yeah I use directly a mesh.

And I look at @miguelemosreverte's branch and I see nothing concerning vertex, he is using material & Ambient Occlusion if I understood correctly, to animate water.

Cordially, Jackblue

[iUltimateLP]

Oh, my bad, he removed the custom render code there. Maybe he sees this and answers you that question :)

[miguelemosreverte]

Oh! Thankfully I have just finished finals season so I am back, just in time! First of all, sure, lets do it!

Whats the hot thing these days? Last time I checked we where working on the water, hacking the Ambient Occlusion parameter to insert some other data I do not remember. As any hack, it was not a nice solution, but it really worked without to much modifications. About deleting stuff, yeah my bad, I did delete everything to do the pull thing from a clean state.

[miguelemosreverte]

@iUltimateLP is correct, I managed to modify vertex data like this:

by copying the data from a vertex to the one of the vertex below it. Basically you take a brick index, right? Well, then you start counting down from its sides, until you hit bottom. For performance purposes, lets say you stop counting at 100. If the brick is 100 far from the floor next to it, then no modifications would be done.

But if it has a neighbor down below, then it will copy the vertex index that it wants, and paste it onto his own. Now its own vertex, and the one of the neighbor, have the same index, and so the GPU will render them as the same. You are not deleting vertices, just pulling them together. But the result is really satisfying because of Scheidecker use of World Texture Coordinates: They, (the textures) do not stretch, but texture information is procedurally created for them.

This can be used for less blocky mountains or for water surfaces, for when you simulate water with the bricks.

EDIT: Forgot to mention, it was not all roses, I recall having trouble with the normals. Because, sure, you can change vertex info, but remember the normals? Those are cooked so that the face number 6 of the brick, the one that should be facing up and you are forcing it to an angle, point up and reflects, well, what an horizontal surface would! Basically you cannot have it both ways, either you have cooked normal angles or you have dynamic voxel angles.

[miguelemosreverte]

About UByte4N, how did you end up using that term? It seems like a really low level data structure! But it make sense, since your approach to complex blocks is to, having the same voxel data as before, have more vertices.

I would suggest the opposite. Voxel resolution could allow in the future for, well, more detailed worlds, and if that happen, you could just make high level objects made up of cubes.

But count me in for the trip!

[SirJackblue]

I'm happy someone heard my prayer !

Thank you for your clarifications, I found the UByte4N here: https://github.com/AndrewScheidecker/BrickGame/blob/master/Plugins/BrickGrid/Source/BrickGrid/Private/BrickRenderComponent.cpp#L132

I am new in the amazing world of rendering, and I made the mistake of believing that ubyte4n had no floating point and I learned later that he was normalize by this doc: https://msdn.microsoft.com/en-us/library/windows/desktop/bb322868(v=vs.85).aspx

So for me if I wanted to do more complex shapes I needed more precision and FBrickVertex struct was only allowed to make brick.

I have not fully understood for the voxel resolution, you mean increasing the accuracy of vertex data x, y, z ? I already create my system to determine the type of my brick and get informations like vertex, index, and normals to draw the shapes.

Finally, here is my Skype (mrjackblue) to make some developing session together if you want to.

[miguelemosreverte]

About the voxel resolution I mean that minecraft proposed to use a different kind of voxel type for certain complex shapes, and the opposite would be the way laser scan represent complex shapes, by using a million of voxels. A middle point that will come at some point in the future is where you still have big cubes, but actually they are made up of a grid of smaller cubes, and that is what allows for detail on complex shapes. That way a stair, like the ones in minecraft, is not a special kind of voxel, but an object made up by a combination of them. That is what I mean when I say that at some point in the future more resolution will be available and then this talk of low level vertex tweaking will be out of the window, because you will be happy with the amount of vertices given by the standard blocks.

Now: This is not the present, where visibility of far away blocks and refreshing of their states is still not a trivial problem. So, yes, we will have to copy the already existing solutions out there. And learn about low level vertex manipulation in the process!

[miguelemosreverte]

You have probably put more thought than me about this vertex problem. So, what are your ideas?

I agree with @iUltimateLP that stairs, for example, just screams Procedural Mesh Component, because as he said, it does not belong to the terrain.

Now terrain, that's another thing. And that I have thought of, and concluded that you do not need extra vertices, but processing power to move them around when LOD says its time to do so.

[SirJackblue]

For me the ideal would be an interface, with differents functions, to know if the type has a complex render. If yes, we override the function which return necessary vertices and indexes. As explained in this post: http://www.blockstory.net/node/59

And after on the rendering loop we check if it's a complex render, if yes we add vertices and indexes, otherwise we draw a simple brick, but I don't think that is as simple as that.

For procedural mesh component, I'm afraid it will be glitchy, that it needs more performance, and the render isn't coherent (procedural mesh is displayed whereas the grid isn't displayed yet) And for the LOD, if we want to save vertices, we can render standard bricks far away, and when the player comes close it renders the correct mesh.

[miguelemosreverte]

Sure. How about this:

Right now, the game sends batches of vertices to the GPU. Those are of a known size. We could add a secondary vertex batch-sender, one that sends batches of exactly the same size, with all the objects that happen to fit inside of it. The render component we know could be renamed as the main grid renderer, and this clone of his, the miscellaneous one.

There are many ideas to talk after this main one is decided, like the priorities of the objects that want to be rendered. But again, no need to talk about that yet.

Over here the objects waiting to be rendered, they all would be complex. Stairs, trees, all static in place and using the voxels system of coordinates.

One last thing, this object, they should have a boolean variable that states whether if not rendered as complex blocks they should not be rendered at all or rendered as normal blocks. This way tree leaves are rendered and complex windows are not so that the hole on the wall is still visible.

[SirJackblue]

What about increase size of batches vertices ?

[miguelemosreverte]

At the end of the day what I am saying is to send, along with the grid data, the objects seen by the player categorized by importance, distance, and complexity, just to name a few. This info would be used alongside the grid data by BrickRenderComponent.cpp after the grid is rendered, and sent to the GPU too, not all but only the amount that can be done without killing the FPS, being the ones chosen and their levels of details thought really well.

[miguelemosreverte]

About increasing the size of batches, you lose control. My solution adds tons of tweaking room.

[miguelemosreverte]

But is can be done, what you say. I mean, using a single batch with different data in it would be cleaner. What you do with that extra data is the important part. Making them separate only adds complexity.

[miguelemosreverte]

So, yeah. Thinking it trough, one big batch, with a separate area for miscellaneous data.

[SirJackblue]

Vertices array on the vertex buffer is a dynamic array, well it doesn't have a limited size, or is there a limitation ?

And because I'm new to rendering, I have no idea how to render all of this :/ If you have any documentation about the subject, don't hesitate to share it !

[miguelemosreverte]

Luckily it truly is not necessary for us mortals to know the insides of the engine.

About the array, is it dynamic? I am going to go check right now.

[miguelemosreverte]

This should be tested, but for now I have to say that even if TArray supports dynamic array size, the data structure used for the voxels, (Grid>Regions>FInt3 Array of Voxel Indexes), could be really rigid.

But, like I said, if it is a pieceof cake to modify, great. Else, it should not be that hard anyway.

[miguelemosreverte]

Quick question to get you thinking: The faces for the bricks are hard-coded as 6. 5 is the top, 6 the bottom. 1,2,3,4 the sides. You see how you would have to modify that so that it supports all the amount of faces needed?

[SirJackblue]

For my own project, complex brick can't be bigger than a normal brick. If it's bigger I use mesh/actor, but we can find a solution in a second time :)

And for the face number, I think we can simply add a function (in the IBrickData) which returns the number of faces for each brick render type.

Example: TScriptInterface<IBrickData> BrickData = UBrickRegistery::GetBrickData(BrickMaterial); int32 FacesNumber = BrickData->GetNumberOfFace();

[miguelemosreverte]

Sounds great!

[miguelemosreverte]

Hey finally finals season ended and now I am going to work on this for a while! I am looking at how minecraft implemented the solution for this problem. Have you done that yet? If so, how did they? Post a link if you have one!

[miguelemosreverte]

The way I see it to move vertices around or even create new ones there are 3 options, really.

That being said, I believe that making a water material that moves the vertices of the blocks makes sense, but doing the same for say, a half block, doesn't seem right, even though is doable.

I would suggest doing like @iUltimateLP , and see how to integrate procedural mesh components with the voxels coordinates transformed to world coordinates as, well, their own world coordinates.

That's what I am going to start doing right now, and @iUltimateLP, if you have something done I would love to check that out.

About minecraft, I believe the translation of the way things work there would be to have an interface, (if the implementation of ProceduralMeshComponent.h is not simple enough) so that each custom object has a constructor function that once called will render the specified object, like a slab of wood, the size of half a block in height.

[miguelemosreverte]

Talking about achievable things with Procedural Mesh Component, check this out: https://garvinized.com/posts/2016/

I have just found the guy, and I love the step-by-step tutorials. The cool thing is, he uses Procedural Mesh Components right off the bat, which in a way is what Minecraft creator actually did. It helps to avoid this very situation of ours, where we do not see clearly what's the next step is.

Say hello to him from me when you contact him! :)

[SirJackblue]

Hey !

Lastly, I was working on an another module of my game, while still thinking about how to make the voxel module.

If you want some informations on how Minecraft is rendering, this link is nice: http://greyminecraftcoder.blogspot.fr/2014/12/block-rendering-18.html

There is only the main idea, I think in order to understand better how it works, you need to reverse engine with MCP and look at the net.minecraft.client.renderer package.

I'm afraid that using the procedural mesh component, I will lose a lot of performance in comparaison with how BrickGame renders. And I think if we change the struct FBrickVertex there will be a way to gain more freedom to draw shapes.

I would like @AndrewScheidecker's feedback on it.

P.S: For sure, I tell him hello from you when I contact him :)

[SirJackblue]

After reading some interesting post, I found my answer for ProceduralMeshComponent performance, I read this answerhub post, with the same question about performance: https://answers.unrealengine.com/questions/412702/procedural-mesh-component-rendering-performance-co.html

Well, at this point BrickGame rendering method is better, and I was thinking how can I improve performance for ProceduralMeshComponent and I found this plugin: https://github.com/Koderz/UE4RuntimeMeshComponent

For me there are 2 possibilties:

• Using UE4RuntimeMeshComponent with PolyVox.
• Add support for more complexe shape to BrickGame, with a more "heavy" FBrickVertex structure (I don't know yet how to make this :/).

What is your opinion about this ?

Also, making a benchmark of this can be interesting, but I think BrickGame would stay the best way in term of performance.

[miguelemosreverte]

Anything that includes using PolyVox is great. It supports Marching Cubes meshes right off, and is really great abstracting the voxel problem away. Besides it really functions as a gateaway to a lot of tools, because coders that use PolyVox also use other tools and so you end up joining that group. The best example of that are the guys that made a port of Cubiquity to UE4, (which is in Github by the way, supported until 4.8 I believe).

About modifying BrickGame, main problem I find is not the difficulty, but what the correct object-oriented implementation of it. To not make obfuscated code. But that will be solved really fast once the objective is achieved and a pull request is made, because Scheidecker would probably know how to make it look presentable. That is why I had interest on the implementation of Minecraft, because its written in a really object-oriented way. Again, besides that we should just do it and then try to fix it.

Another cool idea if we manage to change BrickGame support of complex voxel shapes, would be to make a branch of BrickGame that uses PolyVox, just like the tutorial link. That way you get the best of both worlds. Or not, because on that implementation PolyVox sends the Mesh ready to be rendered, while on BrickGame thats part of the job of RenderComponent. Yeah I dont think PolyVox has a place inside of BrickGame then, as one of the main features of BrickGame as a Render Plugin is to create the meshes.

About the specifics, the only solution I find is to add empty places on the current data structure, ready to be filled. If not used the usual cubes are rendered, but if used then vertices are created on them. It would indeed be a heavier structure, but its the only way to go. Problem is, that's just supporting the complex blocks. That being done now, what is going to be the way to use that support? Hardcoding? You bet that is going to be the option of choice at the beggining, but after a while it has to be made in a correct object-oriented way just so obfuscation is avoided.

[miguelemosreverte]

"add empty places on the current data structure, ready to be filled. If not used the usual cubes are rendered, but if used then vertices are created on them" That does not sounds right, and its because until its done there is no way an explanation that short is going to be the real one. So I am going to work on it, see what happens, and come back with results.

[SirJackblue]

Do you want to find a correct object-oriented implementation for Brick data or for rendering or both of them ? For data, Minecraft uses a Registry design pattern with a Block base class, and child class override functions when it's needed. But I think it would be cleaner with an interface like this: http://hastebin.com/epipevopaf.vala

Sorry for my ignorance, I did not really understand the interest to add some empty places in the data structure. Also, I would like to understand what are doing each FChunkVertexFactory, they render the chunk but how ?

In my mind, we iterate through each x,y,z coordinates, get brick data, look at its render rules (opaque, solid, complex), and add vertices and indices, etc.. consequently. And before we change FBrickVertex to draw more complex things.

[miguelemosreverte]

I would dismiss my now infamous phrase about adding empty spaces, I had just woken up and was drinking some coffee. But the idea is really simple, right now you have 4 vertices per block to play with. If you want complex blocks you must have more information at yout disposal to manipulate. I mean, that can't be wrong, right?

About the snippet of code, yeah, thats what I am talking about. Object-oriented design, yay! Problem is while college makes you respect object-oriented design, practice gives you authority to critize the way its implemented. And right now all I can do is be a big fan of that snippet of code and wait until somebody with more authority than mine comes and defies that idea. Again, that's when Scheidecker feedback would come in handy.

But what I can do is start doing exactly what you said, understanding how exactly are the cubes rendered. That's the way to go for sure.

[miguelemosreverte]

Talking about adding information, see where VertexFactories are initialized in FChunkVertexFactory?

Is only initializing 6 VertexFactories for the 6 faces, if we are going to have more faces that has to be changed.

[miguelemosreverte]

About FChunkVertexFactory, take a look at FLocalVertexFactory from here, is really similar.

https://wiki.unrealengine.com/Procedural_Mesh_Generation

[miguelemosreverte]

Did it!

Right now I am adding exactly 1 primitive, (a triangle made by 3 vertices) to the face 5 of every brick with material 9.

The coordinates of those triangles are always one unmodified coordinate of the brick, one with two substracted from its Z coordinate, and then 0,0,2 as the last vertex coordinate. That number is the responsible of the weird extension of this primitive, where it will extend until the 0,0,2 vertex index inside that Region. That is why one triangle goes one way and the other goes the other way.

[miguelemosreverte]

So, adding extra faces to the blocks, done. Now, about the resolution given, the blocks vertices coordinates are still uint8, in order to construct a face half the height of a normal block you would need to change that.

[miguelemosreverte]

hey I was doing some research, thinkinh about modifying VertexBuffer and guess what? I google'd RHILockVertexBuffer and it appears that the UE4 port of Cubiquity guts are really similar to Scheidecker code

[miguelemosreverte]

Here is their version of FBrickVertex: We could copy them! Instead of using 3 uint8 coordinates they use the Position object. Cool!

struct FColoredCubesVertex
{
    FColoredCubesVertex() {}

    FColoredCubesVertex(const FVector& InPosition, const FColor& InColor) :
        Position(InPosition),
        Color(InColor)
    {}

    FColoredCubesVertex(const FDynamicMeshVertex& other) :
        Position(other.Position),
        Color(other.Position)
    {}

    FVector Position;
    FColor Color;
};

[miguelemosreverte]

The extra primitive, in all of its glory! https://www.youtube.com/watch?v=PKkXbx1VHfg

[miguelemosreverte]

I am going to post a branch with the awful code that results on this, but an explanation would be cool, so here it goes:

We were spot on! TArray, as dynamic arrays, support extra vertices, extra indices, you name it. But the original code has some hardcoded parts that would require modifications.

Usually you have 6 VertexFactories, one per face. Now if you want another face a quick way to do so is to make them 7 VertexFactories, and when you build the Element instead of using a normal FaceIndex you use a FaceIndex of 6, (usually they would go up to 5). And that's it, if there is another hardcoded number to change I may not remember it, but I think it was as easy as that. Again, a branch with the hardcoded abomination will be posted.

[SirJackblue]

Woaw ! Amazing :+1:

Well you need MAX_NUMBER_FACES VertexFactories ? It's not a bad idea to have a VertexFactory used for only drawing some face ?

[miguelemosreverte]

Yeah!

Most importantly, the fact that we have Cubiquity source code to play with is great, since they already altered Scheidecker code to support complex shapes by not having brick integer coordinates but of floating point instead. (About who copied who, Scheidecker repository was created on 2014-04 an theirs on 2014-12 so there is a difference of six months in between.)

[miguelemosreverte]

About that MAX_NUMBER_FACES VertexFactories, I think you are right, now is the time to find if a VertexFactory can be reused, a lot.

[miguelemosreverte]

The Cubiquity guys only use 1 VertexFactory.

The reason BrickGame uses 6 VertexFactories is that having 6 faces, the FaceIndex is then used inside the factory initializer, here:

NewData.TangentBasisComponents[0] = FVertexStreamComponent(&TangentBuffer,sizeof(FPackedNormal) * (2 * FaceIndex + 0),0,VET_PackedNormal);
NewData.TangentBasisComponents[1] = FVertexStreamComponent(&TangentBuffer,sizeof(FPackedNormal) * (2 * FaceIndex + 1),0,VET_PackedNormal);

Cubicuity UE4 port omits this part. So what we could do is leave the 6 factories as usual and add an extra one, used by vertices that will not use their FaceIndex.

[miguelemosreverte]

Hey besides fixing the bug, (now the extra primitive gets rendered correctly) Now there is, besides FBrickVertex, another structure called FDynamicMeshVertex, that supports floating point vertices coordinates.

Integrating this structure inside FBrickChunkVertexFactory by using the strategy design pattern, now there is support for complex shapes. Problem is... The primitives using the new structure are not getting rendered!!!... I will keep working on it until it does so. The code is uploaded to my branch ComplexShapes, if you people wanna take a look.

[miguelemosreverte]

Still no clue. Damn! But different topic, about good design, the Cubiquity project has a different cpp for the VertexFactories. Doing so would take a lot of code from BrickRenderComponent.cpp, so I think after we find how to render those primitives, we do so.

[miguelemosreverte]

Good news! Bugs have been encountered! That means it is creating the mesh:

right here the extra primitives created by the new code can be seen, its color is white by default and so it appears to be in the image.

Here a healthy wireframe can be seen:

And one using the material 9, the hardcoded material that implements the new type of vertices:

These bugs where achieved using these vertex coordinates configurations:

FVector TwoZ(2.0, 0.0, 1.0);
FVector TwoY(0.0, 2.0, -1.0);
FVector Position;

Position.X = SavedVerticesCoordinates[iterator].X;
Position.Y = SavedVerticesCoordinates[iterator].Y;
Position.Z = SavedVerticesCoordinates[iterator].Z;

*FaceVertexIndex_2++ = SceneProxy->VertexBufferComplex.Vertices.Num();
new(SceneProxy->VertexBufferComplex.Vertices) FDynamicMeshVertex(Position);
*FaceVertexIndex_2++ = SceneProxy->VertexBufferComplex.Vertices.Num();
new(SceneProxy->VertexBufferComplex.Vertices) FDynamicMeshVertex(Position + TwoY);
*FaceVertexIndex_2++ = SceneProxy->VertexBufferComplex.Vertices.Num();
new(SceneProxy->VertexBufferComplex.Vertices) FDynamicMeshVertex(Position + TwoZ);

The good thing is, new primitives are being rendered by the new Vertex Factory that should support floating point vertex coordinates. Now why they are acting weird, I dont know yet. I will post the results of using coordinates with actual floating point precision later. Like, intead of 2.0, 0, 0 change it to 2.5, 0.0, 0.5.

[miguelemosreverte]

Changing

FVector TwoZ(2.5, 0.0, 1.5);
FVector TwoY(0.0, 2.5, -1.5);

as vertex coordinates results on this:

which suggest that there was no change at all. Mmmm.

[miguelemosreverte]

Okay that was because I was sharing the indexes of complex vertices and the usual ones, which should not be a problem, and could be done eventually, but not only it fixes the bug but the code looks cleaner after separating the two. Problem is that the complex vertices are yet to be rendered.

[miguelemosreverte]

To be honest I am stuck, I cannot find an reasons for this behaviour, where the usual vertices are rendered and the new ones are not.

[SirJackblue]

Sorry for the late reply, I try to finish as fast as possible a gameplay module before coming back to voxel module, to work on it :)

I looked your BrickGame fork, and I think it seems complicated to put together two Vertex structures, an easier method would be to use FDynamicMeshVertex to draw all (brick and complexe shape) but I don't really know the impact on performance :/

Sadly, I have no idea to unstuck you, but if we can plan a session to reflect on it together I think we will find a solution, and then we will summerize our thoughts in this topic.