Adding option to set custom min_elev

[ansonl]

Chris, I'd like to add an option that lets the user set a custom min_elev in the tile_info dict that is passed to processing for tiles.

When generating 3D models for adjacent regions (dems) that will be fit together, a different min_elev for each generated region will lead to the two regions 3D model having a different height so the top surfaces that are the same DEM height are not the same 3D model height.

I've been getting around this at the moment by manually setting min_elev as 0 for now. I'm a bit busy at the moment and this workaround works so I haven't created a toggleable option for a user selectable min_elev but would like to do so in the future (this issue also helps me not forget the workaround and solution).

[ChHarding]

Hi Anson,

Good to hear from you again. I started experimenting with the masking option you requested a whlle back but then got sidetracked :( Maybe I’ll have time for this again over the holidays. But your current option should be pretty simple to implement.

I also want to let you know that I’ve made some efforts to make it easy to run the jupyter notebook standalone version on Google colab (which I had never used), if you want to try it, just run this GitHub notebook on colab: http://colab.research.google.com/github/ChHarding/TouchTerrain_for_CAGEO/blob/master/TouchTerrain_jupyter_starters_colab.ipynb

Cheers

Chris

On Nov 29, 2022, at 15:05, Anson Liu @.**@.>> wrote:

Chris, I'd like to add an option that lets the user set a custom min_elev in the tile_info dict that is passed to processing for tiles.

When generating 3D models for adjacent regions (dems) that will be fit together, a different min_elev for each generated region will lead to the two regions 3D model having a different height so the top surfaces that are the same DEM height are not the same 3D model height.

I've been getting around this at the moment by manually setting min_elev as 0 for now. I'm a bit busy at the moment and this workaround works so I haven't created a toggleable option for a user selectable min_elev but would like to do so in the future (this issue also helps me not forget the workaround and solution).

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/62, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5NHNHEHWR5B647BCW3WKZVZXANCNFSM6AAAAAASO5KV5U. You are receiving this because you are subscribed to this thread.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

[ansonl]

I tried using gdal_calc.py bundled with QGIS to do the offset masking and it seems to do the job just as well. However the user needs to be more technical to use gdal_calc.

I'll try out the colab if I get a chance!

[ChHarding]

Oh nice! Given the source code (https://github.com/mapgears/pgrastertime/blob/master/gdal_calc.py) how about just using its core code (i.e. w/o all the arg parsing it does) and integrating it into touchterrain? What was the procedure you used with your gdal_cal.py?

On Nov 30, 2022, at 08:45, Anson Liu @.**@.>> wrote:

I tried using gdal_calc.py bundled with QGIS to do the offset masking and it seems to do the job just as well. However the user needs to be more technical to use gdal_calc.

I'll try out the colab if I get a chance!

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/62#issuecomment-1332274540, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5OJ6SSFUIZ6SDYFWVTWK5SCLANCNFSM6AAAAAASO5KV5U. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

[ChHarding]

Anson,

I’ve implemented a min_elev parameter as you requested. I actually did it a while back but I’ve only now released version 3.6 . I did test it a bit but not extensively so its quite possibly still buggy. Could you run it through your use case and see if it’s OK?

Cheers

Chris

On Nov 29, 2022, at 15:05, Anson Liu @.**@.>> wrote:

Chris, I'd like to add an option that lets the user set a custom min_elev in the tile_info dict that is passed to processing for tiles.

When generating 3D models for adjacent regions (dems) that will be fit together, a different min_elev for each generated region will lead to the two regions 3D model having a different height so the top surfaces that are the same DEM height are not the same 3D model height.

I've been getting around this at the moment by manually setting min_elev as 0 for now. I'm a bit busy at the moment and this workaround works so I haven't created a toggleable option for a user selectable min_elev but would like to do so in the future (this issue also helps me not forget the workaround and solution).

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/62, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5NHNHEHWR5B647BCW3WKZVZXANCNFSM6AAAAAASO5KV5U. You are receiving this because you are subscribed to this thread.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

[ansonl]

Chris, I just tested it and it works well, I added the min_elev arg to the standalone file in #68. Thanks.

I did gdalwarp, gdal_calc.py calculations like the below to raise "non river" areas of DEMs. I downscaled the DEMs in advance to a lower resolution such as 500x500 for the entire USA before cropping to state lines or using the printres config option (I set it to max with printres = -1 in TouchTerrain because setting a coarse resolution TouchTerrain smooths the borders of the states too much and the states don't fit together anymore.

#downscale base land dem to 500x500m resolution
gdalwarp -overwrite -t_srs ESRI:102004 -of GTiff -tr 500.0 500.0 -tap ./dems/7-5-arc-second-merged-reproject-102004.tif ./dems/7-5-arc-second-merged-500m-width-reproject-102004.tif -r cubicspline -multi -dstnodata -9999

#crop base land dem by hydro1k/hydrolakes merged dem (cutline is the smaller one)
#or export in QGIS, set export extent using hydro dem
gdalwarp -overwrite -t_srs ESRI:102004 -of GTiff -tr 500.0 500.0 -te -8906000 -3328000 4305500 7828000 -te_srs ESRI:102004 ./dems/7-5-arc-second-merged-500m-width-reproject-102004.tif ./dems/7-5-arc-second-merged-500m-width-reproject-102004-extent-matched.tif -r cubicspline -multi -dstnodata -9999

#create version of land dem with locations that are <=0 (but covered by hydro) raised to 1 elevation
python gdal_calc.py -A ./dems/7-5-arc-second-merged-500m-width-reproject-102004-extent-matched.tif -B ./usa_hydro1k_hydrolakes_merged/usa_hydro1k_hydrolakes_warp_500m_ge_10sqkm.tif --outfile ./dems/7-5-arc-second-500m-width-hydro-raised-above-sea-level-102004.tif --calc="(A > 0) * A + logical_and(A <= 0, B > 0) * 1 + logical_and(A <= 0, B <= 0) * A" --overwrite

#raise nonhydro locations of 500x500 land dem
python gdal_calc.py -A ./dems/7-5-arc-second-merged-500m-width-reproject-102004-extent-matched.tif -B ./usa_hydro1k_hydrolakes_merged/usa_hydro1k_hydrolakes_warp_500m_ge_10sqkm.tif --outfile ./dems/dems-ready-to-cut/7-5-arc-second-merged-500m-width-raised-460.tif --calc="logical_and(B == 0, A > 4) * (A+460) + (B > 0) * A + logical_and(B == 0, A <= 0) * A + logical_and(B == 0, logical_and(A > 0, A <= 4)) * (A + A * 115)"

#raise nonhydro locations of 500x500 hydro fixed (raised) land dem
python gdal_calc.py -A ./dems/7-5-arc-second-500m-width-hydro-raised-above-sea-level-102004.tif -B ./usa_hydro1k_hydrolakes_merged/usa_hydro1k_hydrolakes_warp_500m_ge_10sqkm.tif --outfile ./dems/dems-ready-to-cut/7-5-arc-second-merged-500m-width-hydro-patched-raised-400.tif --calc="(A > 4) * (A+400) + (A<=0) * A + logical_and(A > 0, A<= 4) * (A + A * ((B > 0) * (14) + 100))"

gdalwarp -overwrite -t_srs ESRI:102004 -of GTiff -tr 500.0 500.0 -tap -cutline ./cb_2018_us_state_20m_individual/DE.gpkg -crop_to_cutline ./dems/dems-ready-to-cut/7-5-arc-second-merged-500m-width-raised-850.tif ./dems/7-5-arc-second-clipped-500m/DE.tif -r cubicspline -multi -dstnodata -9999

gdalwarp -overwrite -t_srs ESRI:102004 -of GTiff -tr 500.0 500.0 -tap -cutline ./cb_2018_us_state_20m_individual/DE.gpkg -crop_to_cutline ./dems/dems-ready-to-cut/7-5-arc-second-merged-500m-width-hydro-fixed-raised-800.tif ./dems/7-5-arc-second-clipped-500m-hydro-raised/DE-hydro-raised.tif -r cubicspline -multi -dstnodata -9999

I find that touchterrain tends to generate non manifold models that I need to repair in order to successfully run postprocessing boolean subtractions on. The final finished result is 2 interlocking models that can be dual color printed like below:

Maybe it has to do with my settings so I attached a zip with an example TIF and JSON config if you want to replicate the issue.

MD-no-rivers.zip

[ChHarding]

Anson,

About the non-manifold issue, have you tried setting no_normals to false? Also, if your models outline is a rectangle, the default is to give the entire bottom just 2 triangles, which is obviously non-manifold with any triangle along the bottom. I don’t think that’s the case for you (would be very obvious when you look at the bottom in wireframe ….). How about you send me a small STL that has these issues, maybe there’s a pattern.

Cheers

Chris

On Feb 17, 2023, at 18:13, Anson Liu @.**@.>> wrote:

Chris, I just tested it and it works well, I added the min_elev arg to the standalone file in #68https://github.com/ChHarding/TouchTerrain_for_CAGEO/pull/68. Thanks.

I did gdalwarp, gdal_calc.py calculations like the below to raise "non river" areas of DEMs. I downscaled the DEMs in advance to a lower resolution such as 500x500 for the entire USA before cropping to state lines or using the printres config option (I set it to max with printres = -1 in TouchTerrain because setting a coarse resolution TouchTerrain smooths the borders of the states too much and the states don't fit together anymore.

downscale base land dem to 500x500m resolution

gdalwarp -overwrite -t_srs ESRI:102004 -of GTiff -tr 500.0 500.0 -tap ./dems/7-5-arc-second-merged-reproject-102004.tif ./dems/7-5-arc-second-merged-500m-width-reproject-102004.tif -r cubicspline -multi -dstnodata -9999

crop base land dem by hydro1k/hydrolakes merged dem (cutline is the smaller one)

or export in QGIS, set export extent using hydro dem

gdalwarp -overwrite -t_srs ESRI:102004 -of GTiff -tr 500.0 500.0 -te -8906000 -3328000 4305500 7828000 -te_srs ESRI:102004 ./dems/7-5-arc-second-merged-500m-width-reproject-102004.tif ./dems/7-5-arc-second-merged-500m-width-reproject-102004-extent-matched.tif -r cubicspline -multi -dstnodata -9999

create version of land dem with locations that are <=0 (but covered by hydro) raised to 1 elevation

python gdal_calc.py -A ./dems/7-5-arc-second-merged-500m-width-reproject-102004-extent-matched.tif -B ./usa_hydro1k_hydrolakes_merged/usa_hydro1k_hydrolakes_warp_500m_ge_10sqkm.tif --outfile ./dems/7-5-arc-second-500m-width-hydro-raised-above-sea-level-102004.tif --calc="(A > 0) A + logical_and(A <= 0, B > 0) 1 + logical_and(A <= 0, B <= 0) * A" --overwrite

raise nonhydro locations of 500x500 land dem

python gdal_calc.py -A ./dems/7-5-arc-second-merged-500m-width-reproject-102004-extent-matched.tif -B ./usa_hydro1k_hydrolakes_merged/usa_hydro1k_hydrolakes_warp_500m_ge_10sqkm.tif --outfile ./dems/dems-ready-to-cut/7-5-arc-second-merged-500m-width-raised-460.tif --calc="logical_and(B == 0, A > 4) (A+460) + (B > 0) A + logical_and(B == 0, A <= 0) A + logical_and(B == 0, logical_and(A > 0, A <= 4)) (A + A * 115)"

raise nonhydro locations of 500x500 hydro fixed (raised) land dem

python gdal_calc.py -A ./dems/7-5-arc-second-500m-width-hydro-raised-above-sea-level-102004.tif -B ./usa_hydro1k_hydrolakes_merged/usa_hydro1k_hydrolakes_warp_500m_ge_10sqkm.tif --outfile ./dems/dems-ready-to-cut/7-5-arc-second-merged-500m-width-hydro-patched-raised-400.tif --calc="(A > 4) (A+400) + (A<=0) A + logical_and(A > 0, A<= 4) (A + A ((B > 0) * (14) + 100))"

gdalwarp -overwrite -t_srs ESRI:102004 -of GTiff -tr 500.0 500.0 -tap -cutline ./cb_2018_us_state_20m_individual/DE.gpkg -crop_to_cutline ./dems/dems-ready-to-cut/7-5-arc-second-merged-500m-width-raised-850.tif ./dems/7-5-arc-second-clipped-500m/DE.tif -r cubicspline -multi -dstnodata -9999

gdalwarp -overwrite -t_srs ESRI:102004 -of GTiff -tr 500.0 500.0 -tap -cutline ./cb_2018_us_state_20m_individual/DE.gpkg -crop_to_cutline ./dems/dems-ready-to-cut/7-5-arc-second-merged-500m-width-hydro-fixed-raised-800.tif ./dems/7-5-arc-second-clipped-500m-hydro-raised/DE-hydro-raised.tif -r cubicspline -multi -dstnodata -9999

I find that touchterrain tends to generate non manifold models that I need to repair in order to successfully run postprocessing boolean subtractions on.

Maybe it has to do with my settings so I attached a zip with an example TIF and JSON config if you want to replicate the issue.

MD-no-rivers.ziphttps://github.com/ChHarding/TouchTerrain_for_CAGEO/files/10772504/MD-no-rivers.zip

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/62#issuecomment-1435413990, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5O3AZ3ELK6N2Y7DW2DWYAH2PANCNFSM6AAAAAASO5KV5U. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

[ansonl]

X/Y horizontal non manifold fixed with https://github.com/ChHarding/TouchTerrain_for_CAGEO/commit/ae9c16e057d4cb00711d6c4b431994c6f596bea1