search

gwaldron / osgearth

3D Maps for OpenSceneGraph / C++14
https://www.pelicanmapping.com/home-1/opensource
Other
1.5k stars 781 forks source link

osgEarth 3.1 and osgearth_conv appear to have trouble processing terrain file #1713

Closed keince closed 3 years ago

keince commented 3 years ago

Using osgEarth 3.1 on Windows 10 64-bit, running the utility osgearth_conv.exe on my terrain tif file yields the following results...

**osgearth_conv --in driver GDALElevation --in url osgETerrain.tif --out driver MBTilesElevation --out filename osgETerrain.mbtiles --out format tif --threads 12 [osgEarth] [GDAL] Layer "GDALElevation" Resolution= 0.020232x0.00833333 max=0.00833333 [osgEarth] [GDAL] Layer "GDALElevation" osgETerrain.tif max Data Level: 7 [osgEarth] [TileLayer] Layer "GDALElevation" [srs=WGS 84, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic] [no cache] [osgEarth] [MBTiles] Layer "MBTilesElevation" Database does not exist; attempting to create it. [osgEarth] [osgearth_conv] FROM: { "GDALElevation" : { "driver" : "GDALElevation", "url" : "osgETerrain.tif" } }

[osgEarth] [osgearth_conv] TO: { "MBTilesElevation" : { "driver" : "MBTilesElevation", "filename" : "osgETerrain.mbtiles", "format" : "tif", "profile" : { "num_tiles_high_at_lod_0" : 1.0, "num_tiles_wide_at_lod_0" : 2.0, "srs" : "+proj=longlat +datum=WGS84 +no_defs", "vdatum" : {}, "xmax" : "180", "xmin" : "-180", "ymax" : "90", "ymin" : "-90" } } }

[osgEarth] [osgearth_conv] Calculated max level = 7 Working... [osgEarth] Starting 12 threads [osgEarth] 484 tasks in the queue. 487/248 196% complete, 0m10s projected, 0m-10s remaining Complete. Time = 21.2 seconds.**

When I run the osgearth_conv.exe utility on the same tif file built using osgEarth 3.0, I get the same output except the following is reported...

42346/42346 100% complete

which should be the correct output.

If I use the osgETerrain.mbtiles built with the osgearth_conv.exe from 3.1 in my earth file, I get the following output with osgearth_viewer...

osgearth_viewer --window 100 100 1280 1024 maps_round_all.earth --sky [osgEarth] [Capabilities] Capabilities: [osgEarth] [Capabilities] osgEarth Version: 3.1.0 build 101 [osgEarth] [Capabilities] OSG Version: 3.6.5 [osgEarth] [Capabilities] GDAL Version: 3.2.2 [osgEarth] [Capabilities] GEOS Version: 3.9.1 [osgEarth] [Capabilities] GPU Vendor: NVIDIA Corporation [osgEarth] [Capabilities] GPU Renderer: GeForce GTX 1660 Ti/PCIe/SSE2 [osgEarth] [Capabilities] GL/Driver Version: 4.6.0 NVIDIA 462.30 [osgEarth] [Capabilities] GL Core Profile: no [osgEarth] [Map] Map profile is: [srs=WGS84, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic] [osgEarth] [FileSystemCache] Opened a filesystem cache at "C:\ThriveLite\cache" [osgEarth] [Map] [cache=FileSystemCache; policy=read-write; bin=no] [osgEarth] [Layer] Layer "BATHY" Cache bin is [BATHY-6d23530d] [osgEarth] [GDAL] Layer "BATHY" Resolution= 0.0166667x0.0166667 max=0.0166667 [osgEarth] [GDAL] Layer "BATHY" C:\ThriveLiteMaps\earth.tif max Data Level: 6 [osgEarth] [TileLayer] Layer "BATHY" [srs=WGS 84, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic] [cache=FileSystemCache; policy=read-write; bin=yes] [osgEarth] [MBTiles] Layer "DTED0" Got levels from database 0, 7 [osgEarth] [MBTiles] Layer "DTED0" Profile = [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic] [osgEarth] [MBTiles] Layer "DTED0" Min=0, Max=7, format=tif [osgEarth] [MBTiles] Layer "DTED0" Bounds = SW=179.975,-90 NE=-179.977,84.0042, SRS=WGS84 [osgEarth] [TileLayer] Layer "DTED0" [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic] [no cache] [osgEarth] [engine_rex] Activated! [osgEarth] [TerrainResources] Texture unit 0 reserved for Terrain Color [osgEarth] [TerrainResources] Texture unit 1 reserved for Terrain Elevation [osgEarth] [TerrainResources] Texture unit 2 reserved for Terrain Normals [osgEarth] [TerrainResources] Texture unit 3 reserved for Terrain Parent Color [osgEarth] [TerrainResources] Texture unit 4 reserved for Terrain Land Cover [osgEarth] [RexTerrainEngineNode] Creating 2 root keys. [osgEarth] [MapNodeHelper] Activating logarithmic depth buffer (vertex-only) on main camera [osgEarth] [SimpleSkyNode] Using O'Neil atmospheric lighting [osgEarth] [MapNode] Added extension "sky_simple" [osgEarth] [JobArena] Arena OE.JobArena[]: thread 912 started. [osgEarth] [JobArena] Arena OE.JobArena[]: thread 19996 started.

There is no terrain data being displayed in the osgearth_viewer (or it is being displayed at the lowest level of detail). Loading the "osgETerrain.tif" file also shows no terrain using osgEarth 3.1. The same tif file works perfectly fine in osgEarth 3.0 using its osgearth_viewer.

I wasn't able to attach the tif file here but I do have it if you need it for testing.

The attached text file shows the dt0 files that comprise the tif elevation file...

osgETerrain.txt

gwaldron commented 3 years ago

It would be helpful to get the geotiff. In the meantime what's the output of gdalinfo?

keince commented 3 years ago

I tried attaching the tif file but it wouldn't take.

Here's the output from gdalinfo on the osgETerrain.tif file in question...

Driver: GTiff/GeoTIFF Files: osgETerrain.tif Size is 17796, 20881 Coordinate System is: GEOGCS["WGS 84", DATUM["WGS_1984", SPHEROID["WGS 84",6378137,298.257223563, AUTHORITY["EPSG","7030"]], AUTHORITY["EPSG","6326"]], PRIMEM["Greenwich",0], UNIT["degree",0.0174532925199433], AUTHORITY["EPSG","4326"]] Origin = (-180.025000000000006,84.004166666666677) Pixel Size = (0.020231955406214,-0.008333333333332) Metadata: AREA_OR_POINT=Area Image Structure Metadata: COMPRESSION=LZW INTERLEAVE=BAND Corner Coordinates: Upper Left (-180.0250000, 84.0041667) (180d 1'30.00"W, 84d 0'15.00"N) Lower Left (-180.0250000, -90.0041667) (180d 1'30.00"W, 90d 0'15.00"S) Upper Right ( 180.0228784, 84.0041667) (180d 1'22.36"E, 84d 0'15.00"N) Lower Right ( 180.0228784, -90.0041667) (180d 1'22.36"E, 90d 0'15.00"S) Center ( -0.0010608, -3.0000000) ( 0d 0' 3.82"W, 3d 0' 0.00"S) Band 1 Block=256x256 Type=Int16, ColorInterp=Gray NoData Value=-32767

As mentioned before. this tif file works correctly when using osgearth_conv.exe version 3.0 to convert to MBTiles. It also displays correctly when using osgearth_viewer.exe version 3.0. Neither works when using the same exes built from osgEarth 3.1.

keince commented 3 years ago

How can I get you the osgETerrain.tif file?

gwaldron commented 3 years ago

Try zipping it or putting it on dropbox?

keince commented 3 years ago

I tried the zip file approach and I couldn't drop it below. Perhaps it's too big? 151Mb...

keince commented 3 years ago

How do I use the dropbox?

gwaldron commented 3 years ago

Or try wetransfer, and send the file to support@pelicanmapping.com

keince commented 3 years ago

Just sent it via wetransfer to support@pelicanmapping.com

Let me know if you got it. It should be osgETerrain.tif

gwaldron commented 3 years ago

Got it. I did not test in 3.1, but the file loads correctly in master branch. osgearth_conv also reports 43690 tiles in master branch, as you are expecting. So you might try master.

Also: how did you make the geotiff? Remember that DTED is EGM96-based data so unless you converted it to geodetic heights, you will need to tell osgEarth that it is EGM96 like so

<GDALElevation vdatum="egm96" ...

keince commented 3 years ago

Okay. That's good to know. I made the GeoTiff using the command:

gdal_translate -of GTiff -co BIGTIFF=YES -co TILED=YES -co COMPRESS=LZW osgETerrain.vrt osgETerrain.tif

The osgETerrain.vrt references all the *.dt0 file source data.

I do have the <GDALElevation vdatum="egm96"... included in my earth file.

Here's my entry in the earth file:

<GDALElevation name="DTED0">
    <url>D:/osgEarthTerrainDT0/osgETerrain.tif</url>
    <vdatum>egm96</vdatum>
</GDALElevation>

Works using 3.0 but not 3.1

keince commented 3 years ago

So...I just built the osgearth master branch and ran it with my earth file that loads the osgETerrain.tif file. While there is some elevation displayed now, it is completely wrong. It doesn't resemble anything like what is rendered in 3.0 which is correct.

I'm currently building MBtiles of the data and I will report on those results shortly. It seems to at least be reporting the correct number of tiles...

keince commented 3 years ago

Just ran it with MBTiles generated from the osgETerrain.tif file. Same result. Incorrect elevation rendered.

I'm now building MBTiles from the actual dt0 files referenced in my vrt file, bypassing the osgETerrain.tif file...

keince commented 3 years ago

Just tried loading one simple dt0 file and now there is NO elevation displayed. This, at least did work in version 3.1...

Here is my earth file entry:

<GDALElevation name="DTED0">
     <url>C:\osgEarthTerrainDT0\w112_n40.dt0</url>
     <vdatum>egm96</vdatum>
  </GDALElevation>

Here is the output from osgearth_viewer with OSGEARTH_NOTIFY_LEVEL=INFO set:

osgearth_viewer --window 100 100 1280 1024 "maps_round_all.earth" --sky [osgEarth] [Capabilities] Capabilities: [osgEarth] [Capabilities] osgEarth Version: 3.1.0 build 111 [osgEarth] [Capabilities] OSG Version: 3.6.5 [osgEarth] [Capabilities] GDAL Version: 3.2.2 [osgEarth] [Capabilities] GEOS Version: 3.9.1 [osgEarth] [Capabilities] GPU Vendor: NVIDIA Corporation [osgEarth] [Capabilities] GPU Renderer: GeForce GTX 1660 Ti/PCIe/SSE2 [osgEarth] [Capabilities] GL/Driver Version: 4.6.0 NVIDIA 462.30 [osgEarth] [Capabilities] GL Core Profile: no [osgEarth] [Map] Map profile is: [srs=WGS84, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic] [osgEarth] [FileSystemCache] Opened a filesystem cache at "C:\ThriveLite\cache" [osgEarth] [JobArena] Arena "oe.fscache" concurrency=1 [osgEarth] [Map] [cache=FileSystemCache; policy=read-write; bin=no] [osgEarth] [Layer] Layer "BATHY" Cache bin is [BATHY-6d23530d] [osgEarth] [GDAL] Layer "BATHY" Resolution= 0.0166667x0.0166667 max=0.0166667 [osgEarth] [GDAL] Layer "BATHY" C:\ThriveLiteMaps\earth.tif max Data Level: 6 [osgEarth] [TileLayer] Layer "BATHY" [srs=WGS 84, min=-180,-90 max=180,90 lod0=2,1 vdatum=geodetic] [cache=FileSystemCache; policy=read-write; bin=yes] [osgEarth] [Layer] Layer "DTED0" Cache bin is [DTED0-c17ed814] [osgEarth] [GDAL] Layer "DTED0" Resolution= 2.97521x1.4876 max=1.4876 [osgEarth] [GDAL] Layer "DTED0" C:\osgEarthTerrainDT0\w112_n40.dt0 max Data Level: 0 [osgEarth] [ElevationLayer] "DTED0" : Override vdatum = egm96 (was geodetic) [osgEarth] [TileLayer] Layer "DTED0" [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=EGM96] [cache=FileSystemCache; policy=read-write; bin=yes] [osgEarth] [engine_rex] Activated! [osgEarth] [TerrainResources] Texture unit 0 reserved for Terrain Color [osgEarth] [TerrainResources] Texture unit 1 reserved for Terrain Elevation [osgEarth] [TerrainResources] Texture unit 2 reserved for Terrain Normals [osgEarth] [TerrainResources] Texture unit 3 reserved for Terrain Parent Color [osgEarth] [TerrainResources] Texture unit 4 reserved for Terrain Land Cover [osgEarth] [RexTerrainEngineNode] Creating 2 root keys. [osgEarth] [JobArena] Arena "oe.rex.loadtile" concurrency=4 [osgEarth] [ElevationLayer] "DTED0" : Override vdatum = egm96 (was egm96) [osgEarth] [MapNodeHelper] Activating logarithmic depth buffer (vertex-only) on main camera [osgEarth] [SimpleSkyNode] Using O'Neil atmospheric lighting [osgEarth] [MapNode] Added extension "sky_simple" [osgEarth] [JobArena] Arena "oe.rex.createchild" concurrency=2 [osgEarth] [GPUJobArena] oe.GPUJobArena attached to GC 000001B6ECC7A890

Attached is the dt0 file I'm using: w112_n40.zip

keince commented 3 years ago

image This is how it looks in osgEarth 3.1 using w112_n40.dt0

keince commented 3 years ago

image This is how it looks with osgEarth master branch. Same exact data and earth file. NO elevation rendered.

gwaldron commented 3 years ago

OK. Two things. One, we introduced a bug that made insets break. Pull for the fix.

Secondly, your GeoTiff has confusing extents that are out of bounds. gdalinfo reports:

Upper Left  (-180.0250000,  84.0041667) (180d 1'30.00"W, 84d 0'15.00"N)
Lower Left  (-180.0250000, -90.0041667) (180d 1'30.00"W, 90d 0'15.00"S)
Upper Right ( 180.0228784,  84.0041667) (180d 1'22.36"E, 84d 0'15.00"N)
Lower Right ( 180.0228784, -90.0041667) (180d 1'22.36"E, 90d 0'15.00"S)

osgEarth sees this and sees the west extent at -180.025, and the east extent at 180.0228, which is only ~0.04 degrees wide. You can adjust this by running (for example):

gdal_translate -a_ullr -180 -90 180 84 osgETerrain.tif new_file.tif

Hope that helps.

keince commented 3 years ago

Okay. I will do a pull to get the first fix.

Is there a reason why osgEarth 3.0 didn't break with the extents out of bounds? Did it clamp when extents went out of bounds? I'm just trying to understand why 3.0 works with it and the newer version doesn't...

gwaldron commented 3 years ago

Don't remember, but it probably clamped them, which was convenient but technically the wrong thing to do; newer OE (partially driven by GDAL 3) is more strict about source data.

keince commented 3 years ago

Just tested with the new pull and both the "clamped" tif file and the w112_n40.dt0 work!

Next step is to convert over to MBTiles for faster access.

Thank you!

keince commented 3 years ago

One more comment...

I noticed that on the TileLayer output when running osgearth_viewer with the tif file, it says "srs=unknown".

Is this of any concern? I would have expected "srs=WGS 84"...

Here's the line output in question:

[osgEarth] [TileLayer] Layer "DTED0" [srs=unknown, min=-180,-90 max=180,90 lod0=2,1 vdatum=EGM96] [cache=FileSystemCache; policy=read-write; bin=yes]

gwaldron commented 3 years ago

Don't worry about that, it's just GDAL being ignorant :)

keince commented 3 years ago

Glenn,

Since the issue is resolved, should I mark it as closed or do you do that?

Thanks

Shayne

On 06/08/2021 7:26 AM Glenn ***@***.***> wrote:

Don't worry about that, it's just GDAL being ignorant :)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gwaldron/osgearth/issues/1713#issuecomment-856765651 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ATC464KMGYRSF2IVIWRKBVLTRYK7HANCNFSM46IATTQA .
gwaldron commented 3 years ago

Either way. Thanks Shayne.

keince commented 3 years ago

Thank you for the fix...:)

On 06/08/2021 11:20 AM Glenn ***@***.***> wrote:

Either way. Thanks Shayne.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/gwaldron/osgearth/issues/1713#issuecomment-856951446 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ATC464OSJ5N4UXTZ5CZFMLDTRZGPTANCNFSM46IATTQA .