ChHarding
commented
1 year ago Karl,
Remind me how you’re running TouchTerrain. Are you using a standalone .py or notebook file? Or are you running your own server? I ask b/c running a notebook using Colab the (free) Runtime instance has 12.68 Gb of memory.
Re setting max_cells_for_memory_only to 0 this is supposed to help with low memory hardware but it doesn’t affect a whole bunch of things that still need to held in memory e.g. it has to construct a class called grid (see grid_tesselate.py) to store the geometry of the model. Once constructed, the next step is to create STL or OBJ files from it that I think should go straight to disk but it’s possible that there’s some memory overhead required.
I see that there’s a block just above the def for the grid class that set up profiling and grid itself has a @profile decorator (currently commented out ofc). IIRC I used that profile to figure out in which function/method but this could be reconfigured to instead give you the current memory footprint (I think). But, even if profiling here or elsewhere would point to some memory hogging I honestly don’t know how to “fix” those.
Now granted I haven’t done a thorough analysis of which parts of a lot of memory but I do remember that running a server instance on a virtual PC with 2 Gb was a struggle. It could usually run 1 big job but it would die with 2 or more jobs and so we’re now running it with 4Gb.
I know none of this is a direct solution to you problem but if give me more details about what you’re trying to do I’m happy to dive back into profiling to see if there’s something I could potentially re-write/optimize for saving memory.
Cheers
Chris
On Jan 6, 2023, at 15:30, Karl Schmaltz @.**@.>> wrote:
I am trying to run this on a computer with limited resources and I keep getting crashes because I run out of memory. I saw the arg called max_cells_for_memory_only which from what I understand will use a temp file instead of memory if the cells are bigger than that value. I set that value to 0 so it will always use temp files. When I run something I see it get into the expected if statements for the temp file stuff but when I watch the memory usage of the process it still goes way up, almost to 1 gig in my example.
Is there something else I have to do to disable in-memory processing?
— Reply to this email directly, view it on GitHubhttps://github.com/ChHarding/TouchTerrain_for_CAGEO/issues/65, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEYDF5IJAY4LDPAWVJ54YDDWRCFIHANCNFSM6AAAAAATTPIP5A. You are receiving this because you are subscribed to this thread.Message ID: @.***>
Chris Harding Associate Professor Department of Geological & Atmospheric Sciences Touchterrain.geol.iastate.eduhttp://Touchterrain.geol.iastate.edu
keerl
I am trying to run this on a computer with limited resources and I keep getting crashes because I run out of memory. I saw the arg called
max_cells_for_memory_onlywhich from what I understand will use a temp file instead of memory if the cells are bigger than that value. I set that value to 0 so it will always use temp files. When I run something I see it get into the expectedifstatements for the temp file stuff but when I watch the memory usage of the process it still goes way up, almost to 1 gig in my example.Is there something else I have to do to disable in-memory processing?