search

ChHarding / TouchTerrain_for_CAGEO

Touch Terrain: A python app to create 3D printable terrain models (STL/OBJ) from only elevation data (via Google Earth Engine) or from a local geotiff. Has been used for CNC terrain models. Runs as a web app (http://touchterrain.org), as .py file (standalone.py) or as jupyter notebook. Docker image: https://github.com/ChHarding/TouchTerrain_jupyter_docker
http://touchterrain.geol.iastate.edu
191 stars 44 forks source link

z-scale problem with USGS/GMTED2010 data set #41

Closed SimonScherrer closed 3 years ago

SimonScherrer commented 3 years ago

Thanks for this great tool - I really love it ! I am using the TouchTerrain_jupyter_docker image to create STL files of the Alpine region with the USGS/GMTED2010 DEM as input data. The program works fine, but I have a problem with the z-dimension. I request a vertical exaggeration (z-scale) of 5. According to the logfile (see below), the z-dimension in the output is between 1 and ~94 mm for an elevation range of -4 to 4633 m asl. The map scale is 1:1'247'128. My expectation for the z-dimension extent would be 5*(4637m)/1'247'128 ~19 mm and not the 93 mm in the STL-output. Does this make sense or am I missing something here?

DEM_name = USGS/GMTED2010 trlat = 48.1 trlon = 16.2 bllat = 43.5 bllon = 5.1 printres = 0.4 ntilesx = 4 ntilesy = 3 tilewidth = 180 basethick = 1 zscale = 5 fileformat = STLb no_bottom = False unprojected = False no_normals = True

process started: 18:07:50.558343

Region (lat/lon): 48.1 16.2 (top right) 43.5 5.1 (bottom left) center at [10.649999999999999, 45.8] UTM 32 N , EPSG:32632 lon/lat size in degrees: [11.1, 4.600000000000001] Earth Engine raster: USGS/GMTED2010 GMTED2010: Global Multi-resolution Terrain Elevation Data 2010 URL for geotiff is: https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/c9e352d5f77975a4fe2f4ea838f5595d-f08027df12f33821bd3e5a339654a8ff:getPixels geotiff size: 2.3989439010620117 Mb cell size 479.40833227702973 m, upper left corner (x/y): 184572.20792665644 5352594.029873037 Time to add GPX paths:0.0063588619232177734 full (untiled) raster (height,width) (1119, 1873) float64 elev. min/max: -6.0 4633.0 cell size: 479.40833227702973 m
adjusted print res from the requested 0.4 mm to 0.38441003737319807 mm to ensure correct model dimensions total model size in mm: 720 x 430.15483182060865 Cropping for nice fit of 4 (width) x 3 (height) tiles, removing: 1 columns, 0 rows cropped (1873, 1119) to (1872, 1119) cropping changed physical size from 180 mm x 143.38494394020287 mm to 179.9038974906567 mm x 143.38494394020287 mm map scale is 1 : 1247127.5088262174 Cells per tile (x/y) 468 x 373 Multiprocessing: using all cores (no logging info available while processing) ... ... multi-core processing done, logging resumed

4 x 3 tiles, tile size 179.90 x 143.38 mm

tile 1 1 : height: 4.2274165805133155 - 40.12992028050926 mm , file size: 33 Mb tile 1 2 : height: 4.087094120490997 - 93.99369886336191 mm , file size: 33 Mb tile 1 3 : height: 1.120276394304844 - 77.31537218642355 mm , file size: 33 Mb tile 2 1 : height: 4.267508711948263 - 73.1858826486239 mm , file size: 33 Mb tile 2 2 : height: 1.4610595115019023 - 89.44324194549532 mm , file size: 33 Mb tile 2 3 : height: 1.060138197152422 - 43.217014401000256 mm , file size: 33 Mb tile 3 1 : height: 7.955984803963481 - 73.90754101445296 mm , file size: 33 Mb tile 3 2 : height: 1.020046065717474 - 67.13197080194675 mm , file size: 33 Mb tile 3 3 : height: 1.0 - 42.27484931227898 mm , file size: 33 Mb tile 4 1 : height: 3.325343623226985 - 53.88152136269643 mm , file size: 33 Mb tile 4 2 : height: 1.080184262869896 - 53.420461851194524 mm , file size: 33 Mb tile 4 3 : height: 1.0 - 34.93798925968349 mm , file size: 33 Mb

total size for all tiles: 401 Mb added full geotiff as GMTED2010_10.65_45.80.tif

processing finished: 18:08:34.664949

ChHarding commented 3 years ago

Simon,

Well, it’s entirely possible that something is bugged although I’m more inclined to fault my map scale calculations than the z-scale. I’m happy to step through this and compare values but it would help me if you could give me a single tile, simple case, maybe your area but in a 100 x 100 mm single tile?

Can I ask why you need a x5 scale? That's usually pretty insane and I’ve only used it to bring out something interesting in pretty flat topography such as Iowa :) For naturally high-relief areas, such as the grant canyon I just use x1 and that seems to “look” OK:

Here it is as x 5:

Here’s the Matterhorn and again that looks “natural” to me with x1:

Here’s the Brienzersee, again at x1 (with GMTED2010)

So how come you think you need x5?

Cheers

Chris

On Mar 25, 2021, at 15:32, SimonScherrer @.***> wrote:

Thanks for this great tool - I really love it ! I am using the TouchTerrain_jupyter_docker image to create STL files of the Alpine region with the USGS/GMTED2010 DEM as input data. The program works fine, but I have a problem with the z-dimension. I request a vertical exaggeration (z-scale) of 5. According to the logfile (see below), the z-dimension in the output is between 1 and ~94 mm for an elevation range of -4 to 4633 m asl. The map scale is 1:1'247'128. My expectation for the z-dimension extent would be 5*(4637m)/1'247'128 ~19 mm and not the 93 mm in the STL-output. Does this make sense or am I missing something here?

DEM_name = USGS/GMTED2010 trlat = 48.1 trlon = 16.2 bllat = 43.5 bllon = 5.1 printres = 0.4 ntilesx = 4 ntilesy = 3 tilewidth = 180 basethick = 1 zscale = 5 fileformat = STLb no_bottom = False unprojected = False no_normals = True

process started: 18:07:50.558343

Region (lat/lon): 48.1 16.2 (top right) 43.5 5.1 (bottom left) center at [10.649999999999999, 45.8] UTM 32 N , EPSG:32632 lon/lat size in degrees: [11.1, 4.600000000000001] Earth Engine raster: USGS/GMTED2010 GMTED2010: Global Multi-resolution Terrain Elevation Data 2010 URL for geotiff is: https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/c9e352d5f77975a4fe2f4ea838f5595d-f08027df12f33821bd3e5a339654a8ff:getPixels https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/c9e352d5f77975a4fe2f4ea838f5595d-f08027df12f33821bd3e5a339654a8ff:getPixels geotiff size: 2.3989439010620117 Mb cell size 479.40833227702973 m, upper left corner (x/y): 184572.20792665644 5352594.029873037 Time to add GPX paths:0.0063588619232177734 full (untiled) raster (height,width) (1119, 1873) float64 elev. min/max: -6.0 4633.0 cell size: 479.40833227702973 m adjusted print res from the requested 0.4 mm to 0.38441003737319807 mm to ensure correct model dimensions total model size in mm: 720 x 430.15483182060865 Cropping for nice fit of 4 (width) x 3 (height) tiles, removing: 1 columns, 0 rows cropped (1873, 1119) to (1872, 1119) cropping changed physical size from 180 mm x 143.38494394020287 mm to 179.9038974906567 mm x 143.38494394020287 mm map scale is 1 : 1247127.5088262174 Cells per tile (x/y) 468 x 373 Multiprocessing: using all cores (no logging info available while processing) ... ... multi-core processing done, logging resumed

4 x 3 tiles, tile size 179.90 x 143.38 mm

tile 1 1 : height: 4.2274165805133155 - 40.12992028050926 mm , file size: 33 Mb tile 1 2 : height: 4.087094120490997 - 93.99369886336191 mm , file size: 33 Mb tile 1 3 : height: 1.120276394304844 - 77.31537218642355 mm , file size: 33 Mb tile 2 1 : height: 4.267508711948263 - 73.1858826486239 mm , file size: 33 Mb tile 2 2 : height: 1.4610595115019023 - 89.44324194549532 mm , file size: 33 Mb tile 2 3 : height: 1.060138197152422 - 43.217014401000256 mm , file size: 33 Mb tile 3 1 : height: 7.955984803963481 - 73.90754101445296 mm , file size: 33 Mb tile 3 2 : height: 1.020046065717474 - 67.13197080194675 mm , file size: 33 Mb tile 3 3 : height: 1.0 - 42.27484931227898 mm , file size: 33 Mb tile 4 1 : height: 3.325343623226985 - 53.88152136269643 mm , file size: 33 Mb tile 4 2 : height: 1.080184262869896 - 53.420461851194524 mm , file size: 33 Mb tile 4 3 : height: 1.0 - 34.93798925968349 mm , file size: 33 Mb

total size for all tiles: 401 Mb added full geotiff as GMTED2010_10.65_45.80.tif

processing finished: 18:08:34.664949

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/41, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYDF5NEXPRVVGVX5SHSXBTTFOMXBANCNFSM4Z2CPWUA.

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences

TouchTerrain.geol.iastate.edu

ChHarding commented 3 years ago

BTW here’s your area at x5:

On Mar 25, 2021, at 17:21, Chris Harding @.***> wrote:

Simon,

Well, it’s entirely possible that something is bugged although I’m more inclined to fault my map scale calculations than the z-scale. I’m happy to step through this and compare values but it would help me if you could give me a single tile, simple case, maybe your area but in a 100 x 100 mm single tile?

Can I ask why you need a x5 scale? That's usually pretty insane and I’ve only used it to bring out something interesting in pretty flat topography such as Iowa :) For naturally high-relief areas, such as the grant canyon I just use x1 and that seems to “look” OK:

Here it is as x 5: Here’s the Matterhorn and again that looks “natural” to me with x1: Here’s the Brienzersee, again at x1 (with GMTED2010) So how come you think you need x5? Cheers Chris > On Mar 25, 2021, at 15:32, SimonScherrer ***@***.*** ***@***.***>> wrote: > > > Thanks for this great tool - I really love it ! > I am using the TouchTerrain_jupyter_docker image to create STL files of the Alpine region with the USGS/GMTED2010 DEM as input data. The program works fine, but I have a problem with the z-dimension. I request a vertical exaggeration (z-scale) of 5. According to the logfile (see below), the z-dimension in the output is between 1 and ~94 mm for an elevation range of -4 to 4633 m asl. The map scale is 1:1'247'128. My expectation for the z-dimension extent would be 5*(4637m)/1'247'128 ~19 mm and not the 93 mm in the STL-output. Does this make sense or am I missing something here? > > DEM_name = USGS/GMTED2010 > trlat = 48.1 > trlon = 16.2 > bllat = 43.5 > bllon = 5.1 > printres = 0.4 > ntilesx = 4 > ntilesy = 3 > tilewidth = 180 > basethick = 1 > zscale = 5 > fileformat = STLb > no_bottom = False > unprojected = False > no_normals = True > > process started: 18:07:50.558343 > > Region (lat/lon): > 48.1 16.2 (top right) > 43.5 5.1 (bottom left) > center at [10.649999999999999, 45.8] UTM 32 N , EPSG:32632 > lon/lat size in degrees: [11.1, 4.600000000000001] > Earth Engine raster: USGS/GMTED2010 > GMTED2010: Global Multi-resolution Terrain Elevation Data 2010 > URL for geotiff is: https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/c9e352d5f77975a4fe2f4ea838f5595d-f08027df12f33821bd3e5a339654a8ff:getPixels > geotiff size: 2.3989439010620117 Mb > cell size 479.40833227702973 m, upper left corner (x/y): 184572.20792665644 5352594.029873037 > Time to add GPX paths:0.0063588619232177734 > full (untiled) raster (height,width) (1119, 1873) float64 elev. min/max: -6.0 4633.0 > cell size: 479.40833227702973 m > adjusted print res from the requested 0.4 mm to 0.38441003737319807 mm to ensure correct model dimensions > total model size in mm: 720 x 430.15483182060865 > Cropping for nice fit of 4 (width) x 3 (height) tiles, removing: 1 columns, 0 rows > cropped (1873, 1119) to (1872, 1119) > cropping changed physical size from 180 mm x 143.38494394020287 mm > to 179.9038974906567 mm x 143.38494394020287 mm > map scale is 1 : 1247127.5088262174 > Cells per tile (x/y) 468 x 373 > Multiprocessing: using all cores (no logging info available while processing) ... > ... multi-core processing done, logging resumed > > 4 x 3 tiles, tile size 179.90 x 143.38 mm > > tile 1 1 : height: 4.2274165805133155 - 40.12992028050926 mm , file size: 33 Mb > tile 1 2 : height: 4.087094120490997 - 93.99369886336191 mm , file size: 33 Mb > tile 1 3 : height: 1.120276394304844 - 77.31537218642355 mm , file size: 33 Mb > tile 2 1 : height: 4.267508711948263 - 73.1858826486239 mm , file size: 33 Mb > tile 2 2 : height: 1.4610595115019023 - 89.44324194549532 mm , file size: 33 Mb > tile 2 3 : height: 1.060138197152422 - 43.217014401000256 mm , file size: 33 Mb > tile 3 1 : height: 7.955984803963481 - 73.90754101445296 mm , file size: 33 Mb > tile 3 2 : height: 1.020046065717474 - 67.13197080194675 mm , file size: 33 Mb > tile 3 3 : height: 1.0 - 42.27484931227898 mm , file size: 33 Mb > tile 4 1 : height: 3.325343623226985 - 53.88152136269643 mm , file size: 33 Mb > tile 4 2 : height: 1.080184262869896 - 53.420461851194524 mm , file size: 33 Mb > tile 4 3 : height: 1.0 - 34.93798925968349 mm , file size: 33 Mb > > total size for all tiles: 401 Mb > added full geotiff as GMTED2010_10.65_45.80.tif > > processing finished: 18:08:34.664949 > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub , or unsubscribe . > Chris Harding Associate Professor Department of Geological & Atmospheric Sciences TouchTerrain.geol.iastate.edu

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences

TouchTerrain.geol.iastate.edu

SimonScherrer commented 3 years ago

Hi Chris

My purpose is to create a model for whole mountain ranges in small map scales of 1:200k to 1:1Mio. For these scales it is quite common to use z-scales of 3, 5 or even more (see for example https://www.georelief.de/en/produkt/3d-raised-relief-map-alps-large where a z-scale of 6 is used). I think that your map scale is fine. I also found that things look "ok" when I use larger scales (e.g. 1:25K) and a z-scale of 1.

But the z-dimension seems wrong with z-scale>1. I attached you the zip file of the Alpine region as one tile (100 mm). Interestingly the z-dimension is a factor 5 to large (as in my initial example above). Is: ~10 mm, should be only 2 mm. It seems that the z-dimension output in the STL file is multiplied by z-scale although it should not - in other word it seems as if you scale the z-dimension twice! Maybe you can check this in the code.

Best, Simon

[test_alps_gmted2010_1_tile_100mm.zip] (https://github.com/ChHarding/TouchTerrain_for_CAGEO/files/6210335/test_alps_gmted2010_1_tile_100mm.zip)

ChHarding commented 3 years ago

OK, thanks! Looking into it now, might be a couple of days. In the mean time, if you’re sure about the squared z-scaling you could just use the square root of 5 to get to an effective z-scale of 5 :)

On Mar 26, 2021, at 03:42, SimonScherrer @.***> wrote:

Hi Chris

My purpose is to create a model for whole mountain ranges in small map scales of 1:200k to 1:1Mio. For these scales it is quite common to use z-scales of 3, 5 or even more (see for example https://www.georelief.de/en/produkt/3d-raised-relief-map-alps-large https://www.georelief.de/en/produkt/3d-raised-relief-map-alps-large where a z-scale of 6 is used). I think that your map scale is fine. I also found that things look "ok" when I use larger scales (e.g. 1:25K) and a z-scale of 1.

But the z-dimension seems wrong with z-scale>1. I attached you the zip file of the Alpine region as one tile (100 mm). Interestingly the z-dimension is a factor 5 to large (as in my initial example above). Is: ~10 mm, should be only 2 mm. It seems that the z-dimension output in the STL file is multiplied by z-scale although it should not - in other word it seems as if you scale the z-dimension twice! Maybe you can check this in the code.

Best, Simon

[test_alps_gmted2010_1_tile_100mm.zip] (https://github.com/ChHarding/TouchTerrain_for_CAGEO/files/6210335/test_alps_gmted2010_1_tile_100mm.zip https://github.com/ChHarding/TouchTerrain_for_CAGEO/files/6210335/test_alps_gmted2010_1_tile_100mm.zip)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/41#issuecomment-808042174, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYDF5ID44TVWLLGXV3AP4TTFRCIHANCNFSM4Z2CPWUA.

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences

TouchTerrain.geol.iastate.edu

SimonScherrer commented 3 years ago

Sure, take your time. Yes, a correction seems straight forward. If I'm not mistaken, one has to divide by z-scale to get the correct z-dimension in the current implementation.

ChHarding commented 3 years ago

Simon,

as you predicted I did mistakenly apply the z-scaling twice (in two different files)! Thank you! Here’s your area with the new z-scale of 5:

I’ve updated the server but I’m unclear about how to best make this work for your docker container. The safes way would be to delete the standalone folder (download anything you want to keep!) with

rm -rf standalone

and then run the ./install_touchterrain.sh shell script again which will fetch the new version (3.2.1), then put the stuff you save back (upload).

Alternatively, you could try to run the ./upgrade_touchterrain.sh script, which should just pull the new stuff. However, I’ve not tried that recently and I seem to remember wanting to fix something in that update script. If you get errors let me know.

Cheers

Chris

On Mar 27, 2021, at 11:53, SimonScherrer @.***> wrote:

Sure, take your time. Yes, a correction seems straight forward. If I'm not mistaken, one has to divide by z-scale to get the correct z-dimension in the current implementation.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/41#issuecomment-808761270, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYDF5P74G6UCV2VJIGCBW3TFYEQFANCNFSM4Z2CPWUA.

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences

TouchTerrain.geol.iastate.edu

SimonScherrer commented 3 years ago

Hi Chris, I tried the simple solution with the update script and although there was an error at the end, the new version has been installed and works fine. //Thanks a lot, Simon

ChHarding commented 3 years ago

Simon,

Glad to hear it worked. Re update script, I see that I’m missing a closing double quote which screws up just the last line.

Can I ask you some more details?

Are you in Germany? I’m actually german myself but have lived in the US for 25 years.

Assuming you can print and assemble the Alps model, what is is going to be use for? Teaching something?

Cheers

Chris

On Apr 1, 2021, at 08:36, SimonScherrer @.***> wrote:

Hi Chris, I tried the simple solution with the update script and although there was an error at the end, the new version has been installed and works fine. //Thanks a lot, Simon

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/41#issuecomment-811913147, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEYDF5KESYSXZRWJMF43S5LTGRZFFANCNFSM4Z2CPWUA.

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences

TouchTerrain.geol.iastate.edu

SimonScherrer commented 3 years ago

I am from Switzerland and a map and terrain model enthusiast - climatologist by profession. Built models by hand for 20+ yrs with a friend (see his website at http://www.reliefs.ch/reliefs/index.html). Now trying to go 3D-printing. But have no printer yet. Thinking about a Prusa i3. Any experience with 3D Terrain model printing? Another question: It seems that the available DEMs can have awkward errors (spikes etc.) in remote regions. Do you know a good tool to visually correct erroneous grid points in the output STL files?

ChHarding commented 3 years ago

I am from Switzerland and a map and terrain model enthusiast - climatologist by profession.

Cool! I have family in Freiburg and one of my faculty colleagues (Jaqueline Reber) is originally from Switzerland. I’ve even learned to pick up a few things from when she talks Mundart with her husband :) Built models by hand for 20+ yrs with a friend (see his website at http://www.reliefs.ch/reliefs/index.html http://www.reliefs.ch/reliefs/index.html).

That’s a very cool web site! I had no idea that hand made terrain models were “a thing” :)

Now trying to go 3D-printing. But have no printer yet. Thinking about a Prusa i3.

The Prusa i3 is great. Several people “around me” have it and are very happy with it. The only downside I can see for you is that the build plate is not super-large. But, large 3D printers are really still at the fringe right now and I would not recommend starting with one as a first 3D printer. Any experience with 3D Terrain model printing?

I bit :) I’ve spend way too much time tweaking slicer profiles. I’ve listed some heuristics for terrain printing in our latest paper (free access) in Appendix B: https://www.mdpi.com/2220-9964/10/3/108/htm
Basically, as long as you can get your corners to now warp off the plate (should now be a problem with PLA) you’ll be fine. There are some aspects where tweaking the parameters you can you better quality or the same quality but with less material, e.g. I routinely print with very flimsy infill and that’s fine for my use case. Just use a standard profile for now. Once you get more into 3D printing, I’m happy to talk more about the more nitty-gritty stuff.

Also, have you considered using a CNC router/mill instead? Totally different game (kinda messy with the wood dust) but we know that quite a number of people use TouchTerrain for this and I think their models are generally physically quite large compared to 3D printed models. I’ve had a conversion with a guy who uses a CNC router (at least pre COVID …) for this and I’d be happy to give you his email if you want to get some perspective.

Another question: It seems that the available DEMs can have awkward errors (spikes etc.) in remote regions. Do you know a good tool to visually correct erroneous grid points in the output STL files?

I’ve not run into this. I know that there at least used to be an issue of data acquisition on steep angles (e.g. for the 2000-ish STRM data) but I thought this had been fixed in newer data sets. Could this be an issue with the GMTED 90 m data which is from 2010? Have you done a comparison with the AW3D 30 m data? Or the MERIT 90 m data?

If you want to repair the STL, look at Mesh Mixer (Autocad, free), which supports “manual” manipulation. Another method is to clean up the geotiff from which the STL was made (it’s part of the zip file) and re-run that. QGIS is free and generally good enough these days.

Cheers

Chris

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences

TouchTerrain.geol.iastate.edu

SimonScherrer commented 3 years ago

Hi Chris, thanks for all the information. Have definitely to think and research about the CNC router/mill option. Concerning the "spike/whole errors" in GMTED2010: There are similar problems in AW3D 30 m but the MERIT 90 m data indeed looks better at least for the wonderful Kerguelen Islands. It definitely seems a good idea to check the different data sets in remote regions... //Cheers, Simon