ChHarding / TouchTerrain_for_CAGEO

Touch Terrain: A python app to create 3D printable terrain models (STL/OBJ) from only elevation data (via Google Earth Engine) or from a local geotiff. Has been used for CNC terrain models. Runs as a web app (http://touchterrain.org), as .py file (standalone.py) or as jupyter notebook. Docker image: https://github.com/ChHarding/TouchTerrain_jupyter_docker
http://touchterrain.geol.iastate.edu
210 stars 45 forks source link

Combine Terrain with CityGML #83

Open ThomasPe opened 9 months ago

ThomasPe commented 9 months ago

hey, really love your tool! Lately I was investigating on how to best combine terrain models (geotiff) with CityGML or CityJSON files. Since I'm new to Python I wanted to hear your thoughts on how to best approach this and perhaps contribute back to this codebase.

I tried combining seperately generated STLs (using cjio for the CityJSON model) afterwards with somewhat okayish results but that was a lot of work and error-prone.

I'm wondering if integrating the model generation into the TouchTerrain workflow would be the best approach (similar to how the GPX addition works), but I'm worried it might not be a simple as that as the addition of CityGML would potentially introduce holes to the model (thinking of bridges, for example).

The other approach would be to combine the STLs programatically later, however that would probably be also difficult to properly align all parts when the Geo-Data isn't there anymore.

Is there maybe something obivious I'm missing? Seems like there's a reason why not many tools (only found 1 tbh) can do this kind of merge already.

ChHarding commented 9 months ago

Thomas,

Sadly I know next to nothing about City GML (or DSMs in general). I’m guessing you have building mesh models and want to combine them with a DEM? I assume the issue is to have both the DEM and the building model need to have the same coordinate system and the mm-based STLs you get from TouchTerrain won’t work unless you manually scale and position your building model?

Touchterrain has a way to create STLs where the coordinate are UTM coordinates (search for use_geo_coords) here: https://github.com/ChHarding/TouchTerrain_for_CAGEO#general-processing-parameters

I’ve also written a blogpost here: http://blog.touchterrain.org/2020/03/exporting-terrain-models-with-real.html

This was in the context of using Blender GIS as an alternative path draping method: https://drive.google.com/drive/folders/12b2Ne8O3aHSHB9Oo9_BB6AtgLm2pIzC9

TBH I’ve not had any feedback on any of this and have not used it beyond my examples. But maybe something in there can be used as a start for you.

Or if that’s not what you need, let’s talk …

Cheers

Chris

On Dec 19, 2023, at 12:38, Thomas Pentenrieder @.**@.>> wrote:

hey, really love your tool! Lately I was investigating on how to best combine terrain models (geotiff) with CityGML or CityJSON files. Since I'm new to Python I wanted to hear your thoughts on how to best approach this and perhaps contribute back to this codebase.

I tried combining seperately generated STLs (using cjio for the CityJSON modelhttps://cjio.readthedocs.io/en/latest/includeme.html#) afterwards with somewhat okayish results but that was a lot of work and error-prone.

I'm wondering if integrating the model generation into the TouchTerrain workflow would be the best approach (similar to how the GPX addition works), but I'm worried it might not be a simple as that as the addition of CityGML would potentially introduce holes to the model (thinking of bridges, for example).

The other approach would be to combine the STLs programatically later, however that would probably be also difficult to properly align all parts when the Geo-Data isn't there anymore.

Is there maybe something obivious I'm missing? Seems like there's a reason why not many tools (only found 1 tbh) can do this kind of merge already.

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/83, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5P6PYSVAK3FRCS55VTYKHNKDAVCNFSM6AAAAABA3T76ZCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA2DSMRTGY4TGMI. You are receiving this because you are subscribed to this thread.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

ThomasPe commented 9 months ago

Thanks so much for your input!

I assume the issue is to have both the DEM and the building model need to have the same coordinate system and the mm-based STLs you get from TouchTerrain won’t work unless you manually scale and position your building model?

Exactly this. Source material (geotiff DEM & CityGML) both have the same coordinate system though.

use_geo_coords wasn't in the sample config json so I did miss that option, and it's looking promising. When importing both files to 3D Builder they both also lie on top of each other. Only issue is that I can't get the terrain to scale properly yet, even with tilewidth_scale set to 1.0 the resulting . This is a partial of the config I'm using:

{ "basethick": 1, "bllat": 48.0050717, "bllon": 11.395237, "fileformat": "STLb", "importedDEM": "/workspaces/touchterrain/starnberg.tif", "printres":-1, "tilewidth_scale": 1.0, "trlat": 48.0160216, "trlon": 11.4154585, "unprojected": false, "use_geo_coords": "UTM", "no_normals": false }

ChHarding commented 9 months ago

On Dec 19, 2023, at 17:01, Thomas Pentenrieder @.**@.>> wrote:

Thanks so much for your input!

I assume the issue is to have both the DEM and the building model need to have the same coordinate system and the mm-based STLs you get from TouchTerrain won’t work unless you manually scale and position your building model?

Exactly this. Source material (geotiff DEM & CityGML) both have the same coordinate system though.

use_geo_coords wasn't in the sample config json so I did miss that option, and it's looking promising. When importing both files to 3D Builder they both also lie on top of each other

Only issue is that I can't get the terrain to scale properly yet, even with tilewidth_scale set to 1.0 the resulting . This is a partial of the config I'm using

So the “footprints” over lap in. And y but the z-scale of the terrain is wrong? How sever is it? FYI, you’re giving it a basethick of 1 which for your UTM model will translate into a 10 meter added offset. So if your DEM is just shifted up a bit, try using 0 and see if that fixes it. If you’re not 3D printing the DEM a base thickness should not matter anyway as it’s just there so you don’t end up with super-thin parts (i.e. near holes) in your 3D print.

{ "basethick": 1, "bllat": 48.0050717, "bllon": 11.395237, "fileformat": "STLb", "importedDEM": "/workspaces/touchterrain/starnberg.tif", "printres":-1, "tilewidth_scale": 1.0, "trlat": 48.0160216, "trlon": 11.4154585, "unprojected": false, "use_geo_coords": "UTM", "no_normals": false }

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/83#issuecomment-1863590409, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5PZF4WIJZILUJCTIODYKIMFNAVCNFSM6AAAAABA3T76ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRTGU4TANBQHE. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

ThomasPe commented 9 months ago

I can post a sceenshot of how the models align in 3D Builder later tonight. When I don't pass any tileWidth it defaults to the 80m set in the code, so it seems tilewidth_scale is not doing anything right now in the STL export (at least when manually passing a DEM file).

ChHarding commented 9 months ago

Yes, screenshots would help. Also, 1.0 for tilewidth_scale would seem to make sense as you want a 1:1 model. And I just noticed that for UTM coordinates I don’t actually apply a z-scale any way.

On Dec 20, 2023, at 13:25, Thomas Pentenrieder @.**@.>> wrote:

I can post a sceenshot of how the models align in 3D Builder later tonight. When I don't pass any tileWidth it defaults to the 80m set in the code, so it seems tilewidth_scale is not doing anything right now in the STL export (at least when manually passing a DEM file).

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/83#issuecomment-1865016547, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5OCBUPEJDQMBGTD7DLYKM3TJAVCNFSM6AAAAABA3T76ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVGAYTMNJUG4. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

ChHarding commented 9 months ago

Actually, I think using Github for email may not allow you to insert screenshots (maybe that’s not the case anymore?) Why don’t you cc me at @.**@.> just to be sure?

On Dec 20, 2023, at 13:35, Harding, Chris [GE AT] @.**@.>> wrote:

Yes, screenshots would help. Also, 1.0 for tilewidth_scale would seem to make sense as you want a 1:1 model. And I just noticed that for UTM coordinates I don’t actually apply a z-scale any way.

On Dec 20, 2023, at 13:25, Thomas Pentenrieder @.**@.>> wrote:

I can post a sceenshot of how the models align in 3D Builder later tonight. When I don't pass any tileWidth it defaults to the 80m set in the code, so it seems tilewidth_scale is not doing anything right now in the STL export (at least when manually passing a DEM file).

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/83#issuecomment-1865016547, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5OCBUPEJDQMBGTD7DLYKM3TJAVCNFSM6AAAAABA3T76ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVGAYTMNJUG4. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://touchterrain.geol.iastate.edu/

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

ThomasPe commented 9 months ago

Unfortunately GitHub censors all email addresses so I can't send the screenshots to you (I'm using GitHub in the browser). I'll insert them here, hopefully you'll get them in the email as well. Otherwise I'll add a link to them.

Here's the full config file I used for this:

{
"CPU_cores_to_use": 0,
"DEM_name": "USGS/3DEP/10m",
"basethick": 0,
"bllat": 48.0050717,
"bllon": 11.395237,
"bottom_image": null,
"fileformat": "STLb",
"ignore_leq": null,
"importedDEM": "/workspaces/touchterrain/starnberg.tif",
"importedGPX": null,
"lower_leq": null,
"max_cells_for_memory_only": 1000000,
"no_bottom": false,
"ntilesx": 1,
"ntilesy": 1,
"only": null,
"printres":-1,
"smooth_borders": true,
"tile_centered": false,
"tilewidth": 80, // default, can't be null
"tilewidth_scale": 1.0,
"trlat": 48.0160216,
"trlon": 11.4154585,
"unprojected": false,
"zip_file_name": "terrain",
"zscale": 1.0,
"use_geo_coords": "UTM",
"no_normals": false
}

As you can see by the coordinates both models get projected into the right "area of the world" so to say, considering the big x & y values. However as can be seen by the second screenshot the terrain tile is exactly 80m (I imported as m instead of mm).

image

image

It should by somewhere around 1428m on the longer side (manally calculated). If I put that as tilewidth I get the following result. It looks good in terms of scaling but still is a bit off. Not sure if there are some discrepancies in how touchterrain and cjio (for the city model) translate coordinate systems, I'm looking into what cjio does currently. image

For reference here's what it should look like (aligned roughly manually) image

ThomasPe commented 9 months ago

please disregard my last comment, the touchterrain output wasn't actually UTM as the standalone python script doesn't pass that argument (neither the tilewidth_scale). I'm fixing it locally and can perhaps PR that later.

Seems like 3D Builder only honors the coorinates of the first imported model and then aligns the second close by, so the screenshots are basically meaningless. :D

ChHarding commented 9 months ago

I see. Setting a tilewidth_scale should internally create a tilewidth, so having to calculate that is already unexpected. Would it be possible for you to give me a a) a geotiff and b) a city model of some sorts as either a STL or OBJ where its pretty obvious what a proper “fit” would be. I’m currently working on another issue but after that I would like to have a look at your issue and having something to run through the debugger would be very helpful.

On Dec 20, 2023, at 14:12, Thomas Pentenrieder @.**@.>> wrote:

Unfortunately GitHub censors all email addresses so I can't send the screenshots to you (I'm using GitHub in the browser). I'll insert them here, hopefully you'll get them in the email as well. Otherwise I'll add a link to them.

Here's the full config file I used for this:

{ "CPU_cores_to_use": 0, "DEM_name": "USGS/3DEP/10m", "basethick": 0, "bllat": 48.0050717, "bllon": 11.395237, "bottom_image": null, "fileformat": "STLb", "ignore_leq": null, "importedDEM": "/workspaces/touchterrain/starnberg.tif", "importedGPX": null, "lower_leq": null, "max_cells_for_memory_only": 1000000, "no_bottom": false, "ntilesx": 1, "ntilesy": 1, "only": null, "printres":-1, "smooth_borders": true, "tile_centered": false, "tilewidth": 80, // default, can't be null "tilewidth_scale": 1.0, "trlat": 48.0160216, "trlon": 11.4154585, "unprojected": false, "zip_file_name": "terrain", "zscale": 1.0, "use_geo_coords": "UTM", "no_normals": false }

As you can see by the coordinates both models get projected into the right "area of the world" so to say, considering the big x & y values. However as can be seen by the second screenshot the terrain tile is exactly 80m (I imported as m instead of mm).

image.png (view on web)https://github.com/ChHarding/TouchTerrain_for_CAGEO/assets/4225039/b08f5717-ba81-478e-8a97-b0556b17c7d2

image.png (view on web)https://github.com/ChHarding/TouchTerrain_for_CAGEO/assets/4225039/25a56505-0d04-4ece-8069-3586cbfe757b

It should by somewhere around 1428m on the longer side (manally calculated). If I put that as tilewidth I get the following result. It looks good in terms of scaling but still is a bit off. Not sure if there are some discrepancies in how touchterrain and cjio (for the city model) translate coordinate systems, I'm looking into what cjio does currently. image.png (view on web)https://github.com/ChHarding/TouchTerrain_for_CAGEO/assets/4225039/7cc0f407-a2a5-4f23-bd8e-1d563318447e

For reference here's what it should look like (aligned roughly manually) image.png (view on web)https://github.com/ChHarding/TouchTerrain_for_CAGEO/assets/4225039/70f4a47c-1035-4310-b7f2-6bd8de8d23b2

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/83#issuecomment-1865071255, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5JKD6CFTVDO3SJSCZ3YKNBDVAVCNFSM6AAAAABA3T76ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVGA3TCMRVGU. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

ThomasPe commented 9 months ago

I opened a PR illustrating the issue I had. https://github.com/ChHarding/TouchTerrain_for_CAGEO/pull/84

I'll try to get some demo files that are obivous in terms of positioning

ChHarding commented 9 months ago

Thomas,

I appreciate your work in the pull request but it would help me if you could change it so that the .devcontainer and TouchTerrain_standalone.py are in a separate pull request from the changes you made to TouchTerrrainEartheEngine.py. The reason is that I’d be happy to accept the first 2 but (for now) not the 3. One and I can’t seem to figure out how to do that without current PR. (I actually don’t ahem a lot of experience with git/github when it comes to collaborating, so I’d like to keep things simple …)

Cheers

Chris

On Dec 22, 2023, at 08:53, Thomas Pentenrieder @.**@.>> wrote:

I opened a PR illustrating the issue I had.

84https://github.com/ChHarding/TouchTerrain_for_CAGEO/pull/84

I'll try to get some demo files that are obivous in terms of positioning

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/83#issuecomment-1867778733, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5ORFFQQGPAU7TQ5GVTYKWNE7AVCNFSM6AAAAABA3T76ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRXG43TQNZTGM. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

ThomasPe commented 9 months ago

absolutely, I removed the changes for now from TouchTerrrainEartheEngine.

ChHarding commented 9 months ago

Got it, thanks! Can you explain to me what the .devdockercontainer files do? I don’t really user docker for development. Is this specifically for working with VSCode?

On Dec 22, 2023, at 16:54, Thomas Pentenrieder @.**@.>> wrote:

absolutely, I removed the changes for now from TouchTerrrainEartheEngine.

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/83#issuecomment-1868117113, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5OWYTICBENNLWNDFEDYKYFUBAVCNFSM6AAAAABA3T76ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGEYTOMJRGM. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

ThomasPe commented 9 months ago

Devcontainers do work with GitHub codespaces afaik but are mostly used with VS Code, yes. It basically juist runs a docker image and mounts your repository so even though I'm on Windows I can run & debug the touchterrain code without having to set up any dependencies myself.

ChHarding commented 9 months ago

Wow, that’s pretty neat! If I wanted to try that, what would I need to install? Just the official MS Docker extension?

On Dec 23, 2023, at 16:01, Thomas Pentenrieder @.**@.>> wrote:

Devcontainers do work with GitHub codespaces afaik but are mostly used with VS Code, yes. It basically juist runs a docker image and mounts your repository so even though I'm on Windows I can run & debug the touchterrain code without having to set up any dependencies myself.

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/83#issuecomment-1868377279, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5IMDF4MEKVL6A32SFDYK5IB5AVCNFSM6AAAAABA3T76ZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGM3TOMRXHE. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

ThomasPe commented 9 months ago

I think the Microsoft Dev Container extensions and Docker for Windows using WSL should suffice.