search

ChHarding / TouchTerrain_for_CAGEO

Touch Terrain: A python app to create 3D printable terrain models (STL/OBJ) from only elevation data (via Google Earth Engine) or from a local geotiff. Has been used for CNC terrain models. Runs as a web app (http://touchterrain.org), as .py file (standalone.py) or as jupyter notebook. Docker image: https://github.com/ChHarding/TouchTerrain_jupyter_docker
http://touchterrain.geol.iastate.edu
191 stars 44 forks source link

Standalone Python Library only using one core #74

Open MrMarlin opened 1 year ago

MrMarlin commented 1 year ago

Hi at all

Firstly thank you for developing and maintaining such a great project. I am really enjoying it. My issue is, that although I set the CPU_cores_to_use argument to 8 the script is still using only one core. It also logs using single-core only.

Is there a setting I am missing or what would be the steps to enable the script to use more than one core?

Thanks and have a nice one

ChHarding commented 1 year ago

Sebastian,

A while ago our ISU servers started to have issues when the app was using multi cores (typically b/c users wanted to process tiles) and we just didn’t have the time to figure out what was wrong with it. I should say that multi-processing is very much at the periphery of my Python programming skill. So the solution was to hardcore if to 1 core in TouchTerrainEarthEngine.py:516 https://github.com/ChHarding/TouchTerrain_for_CAGEO/blob/eb026d3590b57bf48eed457aa8367e01d5a22398/touchterrain/common/TouchTerrainEarthEngine.py#L516

Now, I just removed that to see if it wold work inside a jupyter notebook but I’m also getting weird errors that I can’t even debug b/c of running in multi processing mode. So I don’t really know how to even approach this :(

Is this something you have expertise in?

Cheers

Chris

On Mar 12, 2023, at 17:11, Sebastian Lehner @.**@.>> wrote:

Hi at all

Firstly thank you for developing and maintaining such a great project. I am really enjoying it. My issue is, that although I set the CPU_cores_to_use argument to 8 the script is still using only one core. It also logs using single-core only.

Is there a setting I am missing or what would be the steps to enable the script to use more than one core?

Thanks and have a nice one

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/74, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5OUSRZVM7CGXMI4AJLW3ZCZHANCNFSM6AAAAAAVYLULRA. You are receiving this because you are subscribed to this thread.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu

MrMarlin commented 1 year ago

Hi Chris

Thank you for your detailed answer. I thought the multi-core functionality was only disabled for the server. My bad.

Unfortunately I am by no means experienced with multi processing. But if I find some time to peek my nose into it I will give it a go. Maybe I find a quick win.

cheers Sebastian

ChHarding commented 1 year ago

If I get some time I’ll try to isolate a minimal case. BTW the newest version on Github should default to single core and any server version (like ours) should not have that ability anymore to switch to to multi core via user parameters.

On Mar 14, 2023, at 08:08, Sebastian Lehner @.**@.>> wrote:

Hi Chris

Thank you for your detailed answer. I thought the multi-core functionality was only disabled for the server. My bad.

Unfortunately I am by no means experienced with multi processing. But if I find some time to peek my nose into it I will give it a go. Maybe I find a quick win.

cheers Sebastian

— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/74#issuecomment-1468074913, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5IQW2LH5QGE7H2H3ULW4BUV5ANCNFSM6AAAAAAVYLULRA. You are receiving this because you commented.Message ID: @.***>

Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu