Strange square and contour artifacts in imported terrain

[bjfar]

Hi! I am a Godot noob, playing around with this heightmap plugin in Godot 4.1.2. I am seeing some strange artifacts in my imported terrain, and just wondering if it is obvious why this is happening? My heightmap is real-world data collected by satellite observations so it is possible the artifacts are in the data, but I can't seem to see them when I create more straightforward flat image visualisations.

Here are a couple of screenshots to show what I am seeing:

I am saving the data as raw 16-bit ints, scaled from 0 to 32000,. Heightmap is a square, 2^12+1 on each side as the docs seemed to say I should do. Am I doing something dumb there? I tried a few different scalings but it didn’t seem to make any difference.

I'll try a few more things like rotating the data, looking at different regions etc, should help make it clear if the artifacts are really in the data...

[Zylann]

Have you tried with Godot 4.2.2? I don't know what this would be. It looks random as well, which is wonderful (no), it's as if the normalmap calculation was randomly given exceedingly scaled heights at specific locations. I'd need a minimal map to try importing with description of how you import it, to see if I can reproduce it?

[bjfar]

I haven't tried Godot 4.2.2, but I have some more clues. I zoomed way out and observed more carefully as the terrain loaded in, and I can see the terrain increasing in detail in square patterns exactly as above. Looks like the plugin is tiling the data and loading in all the tiles, I guess doing some LOD stuff along the way? I think I might not have noticed this earlier and the terrain got saved in a half-loaded state or something. My machine is not great and this is pretty big terrain (5000x5000) so now that I am sitting here watching it I can see it is taking quite a while to load in and process. I should probably make sure it has finished doing all this before saving the scene and closing I guess? I don't see any progress bar though so it is not clear to me what is happening behind the hood and what I need to wait for.

[Zylann]

The editor will bake normalmaps as a background task in the editor. It's doing this in tiles using a viewport (doing this in GDScript would be atrociously slow), and is non-blocking so you can do things while it's processing (it's also doing that while you sculpt). After you do a big operation on a maximum size terrain, it can actually take a bit of time for all the tiles to be completed indeed. But it usually doesn't take very long. Do the artifacts show up after it's all done? Note that this process is unrelated to LOD, the "tiles" are just sub-regions of the same normalmap covering the whole terrain, that normalmap is saved as a single PNG in the terrain data folder.

[bjfar]

Indeed the artifacts are gone now that I waited long enough for everything to finish. It took a while though, like 10 minutes. Looks lovely now though:

So I guess there is no problem, I just need to be more patient and notice what is happening.

On a minor note, how do the units of height work? I am trying to do the math on xy vs z scaling to get the heights correctly represented and it doesn't seem to make sense if the height int16 base units are equal to xy units. Like this is about 5000x5000 at 200m resolution, so each xy step is 200m, and my heightmap values are scaled from 0-32000 with real-world values spanning about 16000m, and I then left the default plugin height scaling of 0-400. So I expected that meant 0-400 z units correspond to 16000m, the range of my input, i.e. 16000m/400=40m per unit. But that doesn't seem right at all, it would mean I need to scale z (in the HTerrain node settings, where actually it seems to be y for height...) by 5 to match horizontal units. But that seems like way too much looking at the results. So I guess I am wrong about something in there somewhere...

[Zylann]

like 10 minutes

It should not take 10 minutes 🤔 one tile is processed every frame, so at 60fps it should have finished in only a minute. Granted though, it could be faster than that (it's done with normalmap_baker.gd in the plugin folder, which uses 64x64 tiles by default). Maybe your editor's framerate is considerably lower for some reason?

If you left the default vertical scale, it's very likely going to be wrong. By default with a 4097x4097 terrain and no scaling of the node, you will get a 4097x4097 meters wide terrain, and with a height range of 0..400 it means a value of 65536 in the raw file will correspond to 400 meters height. To match your real-world values, I guess you have to do some math. It also depends a lot on what your source range is. If your raw heights are not normalized (i.e the highest peak isn't 65536 in the 16-bit data) then you will have to account for that as well in your calculations. Note also, the plugin assumes unsigned integers (so in 16-bit values, 0 is lowest, 65536 is highest)

[bjfar]

like 10 minutes

It should not take 10 minutes 🤔 one tile is processed every frame, so at 60fps it should have finished in only a minute. Granted though, it could be faster than that (it's done with normalmap_baker.gd in the plugin folder, which uses 64x64 tiles by default). Maybe your editor's framerate is considerably lower for some reason?

Oh quite possibly, I have no real GPU, though the machine has 8 CPU cores. Things are slow while the processing is going on, though I can fly around the terrain in a nice smooth manner once it is all finished.

If you left the default vertical scale, it's very likely going to be wrong. By default with a 4097x4097 terrain and no scaling of the node, you will get a 4097x4097 meters wide terrain, and with a height range of 0..400 it means a value of 65536 in the raw file will correspond to 400 meters height. To match your real-world values, I guess you have to do some math. It also depends a lot on what your source range is. If your raw heights are not normalized (i.e the highest peak isn't 65536 in the 16-bit data) then you will have to account for that as well in your calculations. Note also, the plugin assumes unsigned integers (so in 16-bit values, 0 is lowest, 65536 is highest)

Hmm maybe there are some factors of 2 missing due to unsigned etc, will check my calculations...

[MGilleronFJ]

Oh I think i know why it took so long... it's because in 4.0, Godot did not implement partial texture update, which to this day is still missing. It was added in Godot 3 precisely because it's a critical feature for performant terrain editing. Right now the tiled update doesn't actually work properly, it ends up re-uploading the entire terrain for each tile because the code is meant to use partial update.