Terrain Gaps and Artifacts Introduced After Conversion

[masonr]

I'm running into an issue where it appears that at tile boundaries, the terrain doesn't quite match up. This leads to gaps, pits, and artifacts being introduced into the converted product. The current area that I'm looking at uses two DTED files that I merge together (after converting to TIF).

I've tested a couple sources of DTED/SRTF (Level 1 & Level 2) DT1/DT2 files and also TanDEM-X GeoTIFFs. All sources I tested have this same issue at the same locations on the earth, so I think this rules out an issue with the source of the data itself. Below are my general steps to using the DTED data within Cesium via GeoServer:

# convert DT2 to TIF
gdal_translate {source}.DT2 {out}.tif

# Use GeotiffTranslate to convert to 8-bit RGB TIF
java -jar GeotiffTranslate.jar -i .\path\to\tif -o .\path\to\out 

# Merge TIFs into single file
gdal_merge.py -o {merged}.tif {converted_file_1}.tif {converted_file_2}.tif

# Use the retile utility to prep for use with WMS
gdal_retile.py -v -r cubic -levels 4 -ps 512 512 -co "TILED=YES" -targetDir {output_dir} {input_tifs}

# Add new "ImagePyramid" data store in GeoServer under the "elevation" workspace

# Set terrain provider to GeoServer's new terrain layer using a new Cesium.GeoserverTerrainProvider object

When viewing the area that the terrain includes within Cesium, I observe the gaps and artifacts shown in the screenshots below. Most of the area is perfectly fine and doesn't have these issues, but about 25% of the Western side of the area does. Any advice or things to try would be appreciated.

[kaktus40]

Hello, first you don't need to merge files to use gdal_retile. It's more interesting to use gdalvrt to generate a virtual file that you use as parameter of gdal_retile. Tests with gdalvrt and come back with your result.

[masonr]

Tried using gdalvrt and ended up with the same result (screenshot below). I'm wondering if it's the MGRS -> Lat/Lon coordinate system conversion that's causing this. Or how'd I would even go about diagnosing/fixing it if that turned out to be the case. Edit: I don't think this is the case since the raw DTED files are in WGS_84 coordinate system.

Here's the exact commands I used:

gdal_translate {file_1}.DT2 {file_1}.tif
gdal_translate {file_2}.DT2 {file_2}.tif

java -jar GeotiffTranslate.jar -i {path/to/tifs} -o {path/to/out}

gdalbuildvrt dted.vrt {path/to/out/file_1}.tif {path/to/out/file_2}.tif

gdal-retile.py -v -r cubic -levels 4 -ps 512 512 -co "TILED=YES" -targetDir {tiled-dir} dted.vrt

Then I add store in geoserver using the "ImagePyramid" tool and point to the directory containing all the tiled images. Keep all the defaults. Publish layer. Add as terrain provider in Cesium.

[kaktus40]

don't translate your dted data!

gdalbuildvrt dted.vrt {file_1}.DT2 {file_2}.DT2
gdal-retile.py -v -r cubic -levels 4 -ps 512 512 -co "TILED=YES" -targetDir {tiled-dir} dted.vrt

gdal-retile.py -v -r cubic -levels 4 -ps 512 512 -co "TILED=YES" -targetDir {tiled-dir} dted.vrt

[masonr]

Sorry about that. Just redid it using the method you described. Using the 'grayToColor' geoserver style. Here's the result -

Lots of jagged peaks and gaps in the terrain.

[kaktus40]

did you use bil/dds plugin?

[masonr]

Negative. Tried using it but my geoserver instance wouldn't start up when the java libraries were added. I'll give that another shot now and report back.

[masonr]

Interestingly enough, it looks identical as the first method I tried (in the OP) -

This is using just the gdalbuildvrt and gdal_retile commands.

These are my BIL settings for the terrain layer I'm using -

[kaktus40]

well, did you test with other elevation datas like srtm? What is the precision of your data?

[masonr]

Welp... I figured out the problem. I wasn't grabbing all of the terrain tiles that I needed for the area that I'm looking at, so the gaps and anomalies were a result of missing terrain data for that area. I'm not really sure why it's not all the same elevation for areas with missing data, so that's why I never thought that this would be the problem in the first place.

I was assuming that a file named for example, "W100/N20.DT2" contained the data for W100°-101°, N20°-21°. In reality, this file contains data for W99°-100°, N20°-21°. Complete facepalm moment for me as I wasted many hours trying to debug this and the issue would have been easily spotted had I inspected the files from the start (via gdalinfo).

Everything is working great, now that I have the right data! I really appreciate all of your help @kaktus40 and am truly sorry for wasting your time dealing with my mistake. Your tool has proved extremely valuable for our offline usage. Thanks and sorry again!

[kaktus40]

No problem. Goob luck

[yogimlim]

I am using CesiumJs version 1.89 with @kaktus40 GeoserverTerrainProvider and my code as given below:

Cesium.Ion.defaultAccessToken = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiI3MjA5Zjk0ZC0yM2U0LTQ2NTgtOGQyNi1lNDRlNDU4YmJmYTIiLCJpZCI6NzYwMDksImlhdCI6MTY0NTc4NjYwOH0.****"; var terrainProvider = new Cesium.GeoserverTerrainProvider({ service: "WMS", url: "http://mydomain****/geoserver/JAMMU_KASHMIR/wms", layerName: "JAMMU_KASHMIR:DEM", // xml: xmlGetCapabilities, // proxy: proxy, heightMapWidth: 64, heightMapHeight: 64, offset: 0, highest: 12000, lowest: -500, styleName: "grayToColor", hasStyledImage: true, waterMask: true, maxLevel: 11, formatImage: { format: "image/png", extension: "png" }, formatArray: { format: "image/bil", postProcessArray: function (bufferIn, size, highest, lowest, offset) { var resultat; var viewerIn = new DataView(bufferIn); var littleEndianBuffer = new ArrayBuffer(size.height size.width 2); var viewerOut = new DataView(littleEndianBuffer); if (littleEndianBuffer.byteLength === bufferIn.byteLength) { // time to switch bytes!! var temp, goodCell = 0, somme = 0; for (var i = 0; i < littleEndianBuffer.byteLength; i += 2) { temp = viewerIn.getInt16(i, false) - offset; if (temp > lowest && temp < highest) { viewerOut.setInt16(i, temp, true); somme += temp; goodCell++; } else { var val = (goodCell == 0 ? 1 : somme / goodCell); viewerOut.setInt16(i, val, true); } } resultat = new Int16Array(littleEndianBuffer); } return resultat; } } }); var viewer = new Cesium.Viewer("cesiumContainer"); var scene = viewer.scene;

    var globe = new Cesium.Globe(Cesium.Ellipsoid.WGS84);
    //viewer.scene.globe = globe;

    var ellipsoid = viewer.scene.globe.ellipsoid;
    viewer.terrainProvider = terrainProvider;

after that scene globe being hide. viewer.terrainProvider = terrainProvider;

viewer.scene.globe.terrainProvider = terrainProvider; nothing is working for me... @kaktus40 or anyone ..... please help. thanks in advance

[kaktus40]

Hello, could you show u=your console log?

[yogimlim]

there is no errors in console log

[Violation] Added non-passive event listener to a scroll-blocking event. Consider marking event handler as 'passive' to make the page more responsive. See Cesium.js:90 [Violation] 'requestAnimationFrame' handler took 111ms

[kaktus40]

could you send your capabilities document from geoserver?

[yogimlim]

Geoserver WMS_Capabilities :

This XML file does not appear to have any style information associated with it. The document tree is shown below.

WMS A compliant implementation of WMS plus most of the SLD extension (dynamic styling). Can also generate PDF, SVG, KML, GeoRSS WFS WMS GEOSERVER Aditya Sareen ML INFOMAP PVT LTD Project Manager Work

New Delhi India aditya@mlinfomap.com NONE NONE text/xml image/png application/atom+xml application/json;type=utfgrid application/pdf application/rss+xml application/vnd.google-earth.kml+xml application/vnd.google-earth.kml+xml;mode=networklink application/vnd.google-earth.kmz image/geotiff image/geotiff8 image/gif image/jpeg image/png; mode=8bit image/svg+xml image/tiff image/tiff8 image/vnd.jpeg-png image/vnd.jpeg-png8 text/html; subtype=openlayers text/html; subtype=openlayers2 text/html; subtype=openlayers3 text/plain application/vnd.ogc.gml text/xml application/vnd.ogc.gml/3.1.1 text/xml; subtype=gml/3.1.1 text/html application/json XML INIMAGE BLANK JSON A compliant implementation of WMS plus most of the SLD extension (dynamic styling). Can also generate PDF, SVG, KML, GeoRSS AUTO:42001 AUTO:42002 AUTO:42003 AUTO:42004 AUTO:97001 AUTO:97002 EPSG:4326 .... .... CRS:84 76.69298109266497 77.23230712890629 8.30873019748267 8.89461003000006