search

geo-data / cesium-terrain-builder

A C++ library and associated command line tools designed to create terrain tiles for use in the Cesium JavaScript library
Other
722 stars 344 forks source link

Visible walls at quantized mesh terrain tiles boundary #86

Open xtassin opened 5 years ago

xtassin commented 5 years ago

After experimenting with several settings for ctb-tiles, I keep getting artefacts along terrain tile junctions (see screenshot)

wall1

I managed to reduce these occurrences by using low values (0.2 to 0.5) for -g switch (mesh-qfactor) at the cost of increased tile weight.

Tests were made with several SRTM data sets with similar problem each time.

Aattached, the 1deg SRTM tile used for the screenshot: N43E007.SRTMGL1.hgt.zip

Ccommand line used to generate terrain tiles: ctb-tile -f Mesh -C -N -o /terrain N43E007.hgt

xtassin commented 5 years ago

@ahuarte47, if that may be of any indication, lowering mesh-factor below 0.5 actually reversed the walls but did not make them disappear.

ahuarte47 commented 5 years ago

@xtassin, I think that mesh-factor is not the way to go. I think it has to be a bug in the code, maybe when working with negative heights (I see some of them in the data). But, sorry I can't study it, my apologies.

xtassin commented 5 years ago

@ahuarte47 thank you for the pointer. Do you think you will work on this project again at some point or is there anybody maintaining it at the moment?

ahuarte47 commented 5 years ago

As far I know at the present time nobody maintains this, but it would be very much useful to know if the negative heights are the cause of this issue.

xtassin commented 5 years ago

I would like to keep documenting this issue as I investigate it. Please someone notify me if this is not the right place to do it (as this is related to the master-quantized-mesh pull request)

@ahuarte47 I am wondering if the lod algorithme should takes care of matching vertices at tiles boundaries. As you can see from this screenshot, the walls appear where the mesh is more refined on one neighbour tile than the other.

wireframe_tiles

ahuarte47 commented 5 years ago

Hi @xtassin: I downloaded your SRTM tile and I have verified this issue.

image

image

You are right, the lod algorithm should takes care of vertices at tiles boundaries. I will try to figure out how to fix it.

xtassin commented 5 years ago

Thank you @ahuarte47 for looking into this. I also noted that the "walls" are accentuated when processing a larger set of SRTM tiles aggregated with gdalbuildvrt. This may simply be caused by the LOD algo taking higher altitudes from the surroundings into account. But it could also be a precision issue.

ahuarte47 commented 5 years ago

Hi @xtassin, I think I fixed this issue. If you agree, please, could you test the update?

Before: image

After: image

Now it is 4x slower than old code :-(. For each tile, the LOD implementation reads their four neighbours. This could be optimised using a single tile-cache in the MeshTiler class (At least with the top-border and bottom-border neighbours).

xtassin commented 5 years ago

Thanks @ahuarte47 ! I am struggling a bit to compile the branch but I will let you know as soon as I have a result.

ahuarte47 commented 5 years ago

I am struggling a bit to compile the branch but I will let you know as soon as I have a result.

You can use this dockerfile, it is directly using my branch https://github.com/geo-data/cesium-terrain-builder/pull/64#issuecomment-411071917

xtassin commented 5 years ago

Thank you. It looks like the docker repo has not been updated yet.

xtassin commented 5 years ago

@ahuarte47 I managed to build your branch and process the test SRTM tile provided in this thread. The result is fine and the walls a not there anymore.

However, when I tried to run ctb-tile on the whole SRTM set (as I did before) I got the following error:

Error: The minimum X value is greater than the maximum X value

I run ctb-tile against a .vrt file that aggregate the whole dataset.

The vrt file was generate with the following command to make sure it is cropped to correct bounds and uses proper projection:

gdalbuildvrt -te -180 -90 180 90 -a_srs EPSG:4326 all.vrt *.hgt

This used to work with the latest stable branch.

xtassin commented 5 years ago

I also need to point out that I built CTB against GDAL 2.0.1 (built from source) in order to bypass a compilation error that occurred with GDAL latest.

ahuarte47 commented 5 years ago

Hi @xtassin, Thanks for your tests.

I managed to build your branch and process the test SRTM tile provided in this thread. The result is fine and the walls a not there anymore.

I am glad to read that walls disappeared.

However, when I tried to run ctb-tile on the whole SRTM set (as I did before) I got the following error:

Error: The minimum X value is greater than the maximum X value

You reported same error a time ago https://github.com/geo-data/cesium-terrain-builder/pull/64#issuecomment-410988085 I think this error comes from GDAL when it tries to create tiles of highest zooms (Low levels). May I have a DEM to try fix this error? This old link does not work.

xtassin commented 5 years ago

@ahuarte47. Thank you for your feedback.

I was careful to generate the .vrt file with the -te switch in order to specify bounds. This is how I fixed the issue last time. And this was working before the "walls" fix.

I believe the new way you process neighbour tiles (in order to match vertices) may be trying to access out of bound tiles (-181 deg or something like that).

For now, I managed to start the process by creating the vrt file with a more agressive crop (-te -179 -89 180 90)

I will update you once I have processed the whole world ;)

Thank you so much for your help fixing this wall problem, this is much appreciated.

ahuarte47 commented 5 years ago

Hi @xtassin, thanks you for your feedback!

I have pushed a new commit to speed up the reading of MeshTiler taking care of neighbours. It uses a very simple cache of raster heights in order to avoid the reading of data from the source GDAL dataset as much as possible.

I believe the new way you process neighbour tiles (in order to match vertices) may be trying to access out of bound tiles (-181 deg or something like that).

I am going to look at this. Thanks for your advice.

ahuarte47 commented 5 years ago

I believe the new way you process neighbour tiles (in order to match vertices) may be trying to access out of bound tiles (-181 deg or something like that).

Fixed

BWibo commented 5 years ago

Hey @xtassin @ahuarte47, I just updated the Docker image.

@ahuarte47 It should be possible to setup a webhook to trigger a build everytime you push a new commit to your repo.

ahuarte47 commented 5 years ago

Hi @BWibo, thanks! About the webhook, sorry, I don't have any experience, do I have to execute a git command to raise a specific event to your repo?

BWibo commented 5 years ago

Hey there, this is new for me as well. I believe you will have to setup a webhook in your git repo (Settings -> Webhooks) that points to my Travis CI repo and triggers a build everytime you push to the repo. I will have to provide a token or some kind of access credentials for you I assume. I hope to find time to look into this later this week. I will update you here when I know how it's done.

For now I setup a daily build of the image. You can look up when the last build happened here: https://travis-ci.org/tum-gis/cesium-terrain-builder-docker

xtassin commented 5 years ago

@ahuarte47 thank you so much for your reactivity. I just rebuilt with your latest fix and launched a worldwide SRTM processing. The bounds issue is gone. I can't tell about performance yet but it appears to be much slower than before the tiling fix. I will update you after processing is complete.

@BWibo thank you for stepping in: your docker image makes things a lot easier.

BWibo commented 5 years ago

@ahuarte47 Hey there, I think I found a way to trigger a build of the image every time you push to https://github.com/ahuarte47/cesium-terrain-builder. I made contact with you on Linkedin as I need to share non-public information with you to do this.

You will receive a trigger URL that you need to send a POST to. So you simply need to setup an empty webhook for this URL for the push event (Settings -> Webhooks).

BWibo commented 5 years ago

Thx @ahuarte47, the trigger has been setup successfully, the Docker image should now be updated on every push to https://github.com/ahuarte47/cesium-terrain-builder.

The Docker image is built on Docker Cloud's infrastructure. Depending on the current load, the build process takes roughly 10 to 20 minutes to complete. Check this page for the build status: https://hub.docker.com/r/tumgis/ctb-quantized-mesh/builds

ahuarte47 commented 5 years ago

Thank you very much @BWibo !

xtassin commented 5 years ago

@ahuarte47, I just finished processing a worldwide SRTM dataset. The resulting mesh seems to be fine: no walls or cracks to be seen. I believe it took about twice the time to process the set compared to previous builds. Thank you so much for you help.

ahuarte47 commented 5 years ago

Hi @xtassin, if you agree, you could close this issue.

xtassin commented 5 years ago

I am re-opening this issue as there are still some case where the mesh matching between tiles is failing. I will try to gather more data and find a test case to reproduce the problem.

image

ahuarte47 commented 5 years ago

Hi @xtassin, is it possible that these gaps are in the border of your raster? Are the adjacent tiles in the image outside of your no-data input raster and come them from global Cesium grid? If this is true, I am sorry there is no solution, they are never taken care of them when the mesh tile creation is performed.

Anyway in order to run faster, neighborhood is only considered for tile levels higher than 6 and some gaps can be appear at high camera zooms.

xtassin commented 5 years ago

Here is a srtm file that produces walls http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/srtm_35_06.zip

However, I am not sure this wall is caused by unmatched mesh vertices as before.

The wall (in the screenshot) is visible on the main runway at Agadir airport 30°22'48.9"N 9°32'10.0"W (30.380244, -9.536106)

The wall is "inside" the srtm tile bounds (not at data boundary)

image

image

ahuarte47 commented 5 years ago

Thanks @xtassin , I will have a look