Make LOD terrain infinitely paged

[TokisanGames]

When using a heightmap on VoxelLodTerrain, any values under around 70 get cut off. VoxelTerrain does not have this problem.

This was made with a simple white to black gradient:

Increasing black by 10s, I was able to connect it by making the gradient 255-70.

[Zylann]

Does this happen if you do the same with a custom VoxelStream outputting gradients like this?

[TokisanGames]

No, a custom VoxelStream allows you to go down to ~0 before cutting it off.

[TokisanGames]

Also LOD + custom stream has it's own problems....

Returning cos+sin wave for custom stream:

What?!

Only drawing LOD 0, looks clean, but we get a short distance and 0 or below doesn't draw:

VoxelTerrain, custom stream and smooth:

[Zylann]

VoxelLodTerrain currently only works within a cubic area starting from positive origin, so it's normal if it gets cut below zero (I think VoxelStreamTest doesnt support this yet).

[Zylann]

Doesn't happen here with noise tuned down

Also, careful with handling LOD in custom streams, you have to generate voxels at different scales depending on the LOD parameter. Otherwise, it will make blocks in the distance generate bigger than they should be and create cracks.

I tried a heightmap as well, but it was cut off at 0 because of the same reason I stated earlier:

It cuts at zero.

[TokisanGames]

:) That "lake" shouldn't be there. It's a heightmap with values from 0-255. There are no holes in it!

(If the pixels were transparent would that make legit holes??)

The deepest part of the "lake" is only value 48, The green lines show where 70 is.

VT doesn't create lakes at value 70, it creates a deep basin.

It looks like VLT has offset it's drawing by 70 units so it gets cut off by the bounding cube you mentioned. And since we can't translate a terrain, this means VLT limits the heightmap to 70-255.

[Zylann]

VoxelLodTerrain only cuts at zero, that is verified fine with other generators. What actually happens is that the VoxelStreamImage has hardcoded vertical offset, making it generate height lower than usual, mostly because it was initially written for blocky and converted to SDF for testing. It all boils down to VoxelLodTerrain not being infinite yet / VoxelStreamImage lacking some configuration.

[Zylann]

This issue could be renamed "Make LOD terrain infinitely paged", because currently it's just a big octree with fixed bounds and that's why holes like this appear at the edges (although, it could be fine to still have this option in the future).

[TokisanGames]

Ok

[Zylann]

An early version of paged LOD is availble in the paged_lod branch.

[TokisanGames]

Looks like a great start!

I assume you know about holes all over the terrain at certain LODs, but not others; not LOD0.

But maybe you haven't noticed this issue. It's present in master, but it's different and maybe worse in paged_lod, so I'll put it here:

Ridges appear on flat terrain where lower LODs sections connect with higher LODs. I only notice this when moving on a flat plane.

The ridge is caused by meshing a lower LOD about a block higher than the plane it was on. The ridge shouldn't exist at all, and the lower LOD mesh should be lower on the Y axis. This might be a DMC limitation, but paged_lod does it differently than master so maybe not.

On master the biggest ridge appears about 25 blocks out and a smaller one is closer around 13 blocks away, and another much farther in the distance.

On paged_lod a large one appears about 15 blocks out, instead of 25. It's good that there's only one ridge instead of three, but since it's closer, it becomes very noticeable on a flat, textured terrain when moving.

For some perspective on the situation, on master (left) the middle ridge aligns with after the hill on the right. On paged_lod (right), the ridge is aligned with the start of the hill. That's quite a large distance closer and even though its only a few pixels change on screen, I didn't notice it until using paged_lod because it's "so much closer" psychologically.

[Zylann]

@tinmanjuggernaut I opened https://github.com/Zylann/godot_voxel/issues/60

About holes, the've been there since LOD was a thing. I never really implemented a solution to completely hide them, so far they are only attenuated with marching square skirts.

[Zylann]

I think I forgot to assign split_scale properly when creating the grid of octrees, which might explain the difference with master. Doing this now. Edit: pushed.

[TokisanGames]

Re: Split_scale, yes it looks like before and is much more subtle now. Thanks! I will discuss anything further on the other thread.

[TokisanGames]

Re: infinite paging:

Looking good so far. A definite major improvement.

I noticed the number of LODs changes view distance and chunk size, and thus (apparent) paging performance.

On master, the heightmap repeated in a 2x2 grid. I can see from one corner to the other, giving a view distance of 2896. This is great and around where I would expect to configure a game to be. VT is too short for smooth terrains. My GTX1060 runs this at 350fps, so plenty more headroom to abuse.

On this branch, I you default to 4 LODs and the view distance becomes much smaller, say around 1300. The paging works pretty well. It loads in about 10-20% of a heightmap at a time. Overall it looks and feels a lot like VT with a very far view distance and at least 4x faster.

Increasing the LODs, I'm surprised to see it also affects view distance and (apparent) paging speed and chunk size!

At LOD6 I get the 2x2 grid. The chunks have a width of 512! It takes some time to load (apparently), then the whole strip, half a heightmap wide appears all at once.

At LOD7, I still have the 2x2 grid. The chunk width is now 1024; the whole heightmap. The whole strip loads at once after some time (apparently). I can be racing off the edge before it finally builds. Then I can pop back and forth over the edge and back, and the whole new heightmap strip will appear and disappear depending on if I'm over it or not. And the heightmap row on the opposite side of the world will be disappearing and reappearing, opposite.

At LOD8 I got a 4x4 grid, and the width of two heightmaps reloaded at once. It does the same thing. It doesn't display until I cross the line into empty space. Then it appears fast.

I thought it was loading, but now I think that it's already there and built, but for some reason isn't displaying until I've crossed the empty border. Thus I've added (apparently) above.

I think if you've already built the terrain in the background, then you should display it as long as it is within view distance. Or maybe dispense with view distance and just use camera view distance so there is only one variable.

Also, if the strips that loaded in were smaller or a fixed size regardless of LOD, there's less chance the user will be waiting for them to load. Hmmm, Is your LOD8 2048 in size???!! If that's the case, what about just loading more, so there is a radius of 3-4 heightmaps around the player instead of 2?

At "normal speeds" (12) it looks totally fine. But when I'm flying or running at super speeds (100) that's when I ran into these issues. And for a game, I'd want to be able to configure view distance and/or loading distance around the player to account for my player speed.

[Zylann]

I think there is a new terminology to be used here: VoxelTerrain was simply made of blocks, themselves made of voxels. VoxelLodTerrain is now made of octrees (That's what they are in code), themselves composed of blocks of varying LOD, themselves made of voxels. Then, those octrees are streamed just like VoxelTerrain does, it almost supersedes it.

LOD does not change view distance (at least, that's not its goal). Increasing LOD counts increases the size of the "octrees" this terrain will create, and the amount of mipmaps to support within the save file. That means it will also create less octrees, because the terrain attempts to fill in a fixed area within the defined view distance. If an octree is bigger than view distance itself, you'll get larger resulting view distance, but that's only because the terrain has no choice^^ (it can't "cut" half of an octree, these are indivisible).

If LOD is too big while view distance is too short, you will then get these odd large chunkings where you got only 2x2, so it's a matter of tuning the parameters to what fits better for you. If you use such a big LOD it means you should increase view distance to not see them pop around too close.

It's possible that I missed a 0.5 factor to translate that view distance into a chunk area, which probably shows the most at extreme chunk sizes. However you are still expected to tune this view distance to allow the terrain to show these chunks. The idea is, with a given view distance, the terrain will always aim to fill blocks within that radius, and maybe a bit beyond due to "fitting cubes in sphere". You could easily make a script that copies camera far clip into view distance.

[TokisanGames]

LOD does not change view distance (at least, that's not its goal).

Yes, what I should have said is LOD Count changes the size of the visible terrain. And I was confusingly using "view distance" for both the VLT variable (which is not currently settable in the editor nor gdscript) and also for the camera far clip, which is set to 8192.

---

I think there's just a bug in paging and it's probably an easy fix that will address everything I brought up.

Here's what happens at each LOD:

1. Unusably slow, I didn't even wait for it.
2. Has big holes; blocks don't even load and is slow.
3. Has big holes as above. Speed is fine. Looks/acts just like VT except for the holes.
4. Looks fine, pages normally around the player.
5. Looks/pages normal, but I'm a little off-center. -X and -Z are further, +X/+Z are shorter (and maybe +/-Y too, but I couldn't see).
6. Same as above. Pages fine, but I'm way off-center now.
7. Wayyyyy off-center. Pages fine in -X/-Z. I get the current octree (large section?), plus one more ahead. Great! In +X/+Z there's nothing ahead. It doesn't page in until I cross the edge.
8. Same as above, but even further off center. Doesn't matter if I'm flying or on the ground.

Here is LOD 7:

Possible solution: If paging in +X and +Z (and +Y?) were fixed, then I should have the current octree, plus one additional octree all around me. If I turn around, it's already there. And as I move, even quickly, the next one will always be loading ahead of me. That would be great.

Changing the LOD count effectively means changing the size of the visible terrain. And VoxelLodTerrain.View_distance is not currently settable.

So how about changing the VLT user parameters like this:

LOD Count -> Map Size or Terrain Size.

• 1-3 are not available.
• 4 -> is renamed to 1250
• 5 -> 2048
• 6 -> 3072
• 7 -> 4096
• 8 -> 5120 ... (or whatever the real numbers are)

view_distance goes away

Now as a user, the camera far clip and the Terrain Size are in the same units and mean something to me. I don't think LOD 1-8 means anything to a user. And I can cull with my camera far clip for performance if needed.

What do you think?

[Zylann]

LOD 1 is slow because if you keep default view distance, it's gonna try to perform like VT, but with 8 times more chunks, which is of course really slow.

For anything regarding paging holes:

It's possible that I missed a 0.5 factor to translate that view distance into a chunk area

I need to do some more testing on my end to find the issue. It's not that things are not available or not workable, they are supposed to work correctly, there is a bug there really.

Changing the LOD count effectively means changing the size of the visible terrain. And VoxelLodTerrain.View_distance is not currently settable.

But it shouldn't, that's the point^^ if it currently does, that's part of the bug. It shouldn't be renamed to terrain size because that's not what it means. A more accurate description is changing "chunk size" or "octree size" (although block size is another similar setting that also exists and can be changed too under some specific conditions). Also view_distance should be settable, maybe I forgot to bind it.

Also, last but not least: If you are using VoxelStreamRegionFiles, delete your save before testing different parameters, otherwise you'll have quite unexpected results :)

[Zylann]

I just pushed fec6e624f6682733d3a47fa4270c07ab253d0547, which corrects the octree region so that it is better centered on the viewer. You should still adjust view_distance accordingly, but there should not be a "walking across world edge" situation anymore (unless it's really slow to load but you can check that with VOXEL_DEBUG_BOXES enabled).

[TokisanGames]

Really nice work. Infinity has been achieved! Or at least far clip has been.

I kept ratcheting up the numbers. My player normally walks at speed 12. I set it to 200, 1000, 2000, and eventually 10,000! I set view distance to 8192, and LOD up to 10 and still maintained 150-200fps on heightmap and 200+ on 3D noise. Amazing!

On the heightmap ground, I moved too fast at 1,000 for the LOD system to keep up. I kept hitting the elevated ground mentioned in #60, even when traversing the flat, continuous heightmap edge. So I had to keep flying.

Then I set the noise height to 1000 with a high period (200) and wow! Then a height of 10,240, and Oh My God! It's so incredible! You have to see it.

We really need occlusion culling, but bumping the LOD to 16 allowed me to get decent performance. I'm going to put this in my next video.

So at the high speeds and far distances #59 black/dark patches and the holes in the mesh become very apparent. But the paging system looks solid. I think we could close this issue unless you want to do something else with it?

[Zylann]

bumping the LOD to 16

That's insanely high xD about 65 kilometers forward (you'd need a far clip of 65,536, which is the size of a Minetest world), I don't think Godot can render that far and not have z-buffer precision issues :D

If it's ok for you after these tests I think it can be closed.