search

heremaps / tin-terrain

A command-line tool for converting heightmaps in GeoTIFF format into tiled optimized meshes.
MIT License
579 stars 126 forks source link

dem2tin failure w/o actionable error msg #44

Closed moosehill closed 5 years ago

moosehill commented 5 years ago

I downloaded tin-terrain today and built the Docker image on my Mac.

When I run dem2tin, I see the following failure:

root@f481e479e8a9:/home# tin-terrain dem2tin --input /data/LMI_Haedarlikan_DEM_16bit/LMI_Haedarlikan_DEM_16bit-Mercator.tif --output /data/LMI_Haedarlikan_DEM_16bit/test.obj subcommand_dem2tin Opening raster file /data/LMI_Haedarlikan_DEM_16bit/LMI_Haedarlikan_DEM_16bit-Mercator.tif with GDAL... reading raster data... Killed root@f481e479e8a9:/home#

My dataset:

root@f481e479e8a9:/home# gdalinfo /data/LMI_Haedarlikan_DEM_16bit/LMI_Haedarlikan_DEM_16bit-Mercator.tif Driver: GTiff/GeoTIFF Files: /data/LMI_Haedarlikan_DEM_16bit/LMI_Haedarlikan_DEM_16bit-Mercator.tif Size is 56541, 39914 Coordinate System is: PROJCS["WGS 84 / Pseudo-Mercator", GEOGCS["WGS 84", DATUM["WGS_1984", SPHEROID["WGS 84",6378137,298.257223563, AUTHORITY["EPSG","7030"]], AUTHORITY["EPSG","6326"]], PRIMEM["Greenwich",0, AUTHORITY["EPSG","8901"]], UNIT["degree",0.0174532925199433, AUTHORITY["EPSG","9122"]], AUTHORITY["EPSG","4326"]], PROJECTION["Mercator_1SP"], PARAMETER["central_meridian",0], PARAMETER["scale_factor",1], PARAMETER["false_easting",0], PARAMETER["false_northing",0], UNIT["metre",1, AUTHORITY["EPSG","9001"]], AXIS["X",EAST], AXIS["Y",NORTH], EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs"], AUTHORITY["EPSG","3857"]] Origin = (-2764161.621573575772345,10052026.112180622294545) Pixel Size = (23.488512876275621,-23.488512876275621) Metadata: AREA_OR_POINT=Area DataType=Generic Image Structure Metadata: INTERLEAVE=BAND Corner Coordinates: Upper Left (-2764161.622,10052026.112) ( 24d49'51.19"W, 66d37'55.54"N) Lower Left (-2764161.622, 9114505.609) ( 24d49'51.19"W, 63d 3'29.08"N) Upper Right (-1436097.615,10052026.112) ( 12d54' 2.46"W, 66d37'55.54"N) Lower Right (-1436097.615, 9114505.609) ( 12d54' 2.46"W, 63d 3'29.08"N) Center (-2100129.618, 9583265.861) ( 18d51'56.83"W, 64d54'16.26"N) Band 1 Block=56541x1 Type=Int16, ColorInterp=Gray NoData Value=-10000 Metadata: BandName=Band_1 SourceBandIndex=0 root@f481e479e8a9:/home#

Since I downloaded rather than cloned, not much here. But I did this today:

root@f481e479e8a9:/home# tin-terrain version name: tin-terrain copyright_notice: Copyright (C) 2018 HERE Europe B.V. git_description: git_hash: build_timestamp: 20181130T2116Z root@f481e479e8a9:/home#

I've tried the verbose switch and the log option, but there is no additional information to help me decide what to try next. Any suggestions would be much appreciated.

adrpar commented 5 years ago

Hi @moosehill,

thank you for reaching out and trying tin-terrain. I would like to understand more about the reason for tin-terrain to be killed by the OS.

The dataset you are using, is it this one I found on this Icelandic page? https://vefthjonustur.wordpress.com/thrividd/ ?

Second, how much memory does your machine have?

I suspect that tin-terrain runs out of memory, but I want to test this first and eventually rule that possibility out, before we start digging deeper.

Thanks for providing this information.

moosehill commented 5 years ago

Hi @adrpar,

Yes, that's the dataset, but I also used gdalwarp per the instructions in the README to convert to Mercator.

My system has 32GB. I watched the Activity Monitor while running tin-terrain and "memory used" never exceeded 24GB with memory pressure shown as about 30%.

Thanks for looking at this.

adrpar commented 5 years ago

I was able to reproduce the issue and it is lies in how Docker runs on Mac. The VM that contains Docker (docker-machine) only has 2GB memory in its default mode. Therefore tin-terrain runs out of memory quite fast.

You can increase the memory by following these instructions here: https://docs.docker.com/docker-for-mac/#advanced

Or you try compiling tin-terrain natively on the Mac without Docker. If you need help with that, let me know.

moosehill commented 5 years ago

@adrpar Thanks for the additional information -- I've increased Docker memory to 27GB and it still fails, but it is getting much further than previously. I will continue to play with Docker settings to see if I can get it to complete. (Also, I need new glasses because I thought the dataset was 5000x4000 when it is actually 100X larger than that. Now I understand why you immediately suspected memory was the problem.)